My question relates to hardware configuration for DW environments. We have a two terabyte DW that is in need of improved data integrity. One of the data issues is that DW warehouse changes to correct legacy-sourced data need to occur to the production DW because neither a quality or full model of the DW exists it’s a cost issue. My question is what is "recommended" related to the hardware configuration for a "typical" production environment. In the client/server development arena, I am used to the classical Production, Quality, Development environments with versioning etc. However, since we are "fixing" so many data integrity issues getting the data correct in "one" environment [i.e., production] is challenge enough.
What is typical? Are there research sources you can point me to?
Clay Rehm’s Answer: There is no typical configuration. I have seen data warehouse environments match operational environments and have seen everything in between. What works for another company may or may not work for you.
You should have a development region for your unit and system testing of the ETL procedures. As soon as specific ETL procedures have passed your system tests, move the associated data and ETL into Acceptance for your end users to test and validate. However, due to other pressures, you may wish to have two regions for now a development/quality region and a production region until you can get a handle on your data quality issues.
There will be dirty data in your production data warehouse. However there should not be dirty ETL processes in the data warehouse. Additionally, you must create a valid data model of your data warehouse. I am not sure how you can support this warehouse without it. Exactly what kind of cost would be incurred to get a decent model of your data warehouse?
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access