The Northwestern Medicine
Recent Posts
-
ETL Assistant – Management Web UI Gallery
1Gallery of screenshots from ETL Assistant Manager – the ASP.NET MVC 3 web console for our ETL automation toolkit built on top of Cozyroc’s amazing SSIS components. This should give you a sense of exactly how quickly we can now integrate external OLEDB data sources using Microsoft SQL Server 2008. Point, Click, Consume. Easy as 1, 2, 3. Continue reading
-
The EDW is Hiring Data Architects!
Interested in joining the team? We’re looking to hire Data Architects to help support and extend the EDW. The Data Architect is responsible for developing information architectures and complex data models, and approves the modification, and new implementation of database systems. This individual works closely with end users and IT development staff to ensure that new and existing data models and databases are consistent with approved data architecture standards. The Data Architect must have expert level knowledge and experience in data modeling and database design. Continue reading
-
Monitoring Oracle GoldenGate Replicats on Windows
Use PowerShell, regular expressions, and Microsoft SQL Server to build a simple Oracle GoldenGate replicat monitoring tool on Windows. This will help you produce a convenient relationally-oriented representation of replicat status and replicat lag, then extend it to communicate that information per mapped table. Continue reading
-
Using the MERGE Statement in SSIS Via Stored Procedure
(this is a modified version of the article I wrote for SQLServer.com published on 1/23/012, ‘Using the MERGE Statement in SSIS Via A Stored Procedure‘) Background In our Enterprise Data Warehouse (EDW) within the Northwestern University Biomedical Informatics Center (NUBIC), … Continue reading
-
NUBIC’s FedEx Day
NUBIC decided to do a FedEx Day (following the Atlassian model). It’s our first FedEx Day, and we had about 1/3rd of our staff build or collaborate on projects between the NUBIC software development and EDW groups. There was also unanimous … Continue reading
-
ETL Assistant – Getting Error Row Description and Column Information Dynamically
SSIS does a fine job at letting you manage “garden path” ETL, but many face the challenge of how to manage row failures. Which row failed? Why did it fail? Error row handling is a central part of any development task and usually winds up representing a significant chunk of your time and code. In this article we’ll step you through how to overcome SSIS’s design-time-only availability of error row information by creating a runtime dynamic error row handler using CozyRoc’s tool kit for SSIS. Continue reading
-
ETL Assistant – Using CozyRoc’s Parallel Loop Task
In our continuing series on building your own templates, reusable “point-and-click” ETL solution using CozyRoc’s components for Microsoft SSIS, we’ll be stepping into the “Parallel Loop Task”. Microsoft’s default loop task is serial. If you ever have need to parallelize execution of SSIS tasks, overcoming this can be quite a challenge. With the Parallel Loop Task you can, with a few quick clicks, enable parallel execution of tasks and packages to maximize the utilization of your available resources. Continue reading
-
Mr. Can’t-Know-It-All
Okay, let me start by saying that I struggle with the cliché that there are no stupid questions. I say that partly because I do believe there are stupid questions out there, typically asked by people who just don’t get … Continue reading
-
Our Journey to Meaningful Use Certification – Part One
The Northwestern Medical Enterprise Data Warehouse received meaningful use certification for both of our Inpatient and Outpatient modules back in June 2011. Since then a lot of people have been contacting me to learn about our certification experience and I … Continue reading
-
Our Fundamental Principles of Data Warehousing
I think many data warehousing shops get off on the wrong foot in defining their architecture. When we set out to create the Northwestern Medical Enterprise Data Warehouse, we identified three principles that govern everything we do. These were not … Continue reading