I am the quality assurance (QA) manager for the Data Management Services department of the Bank of Montreal. We build data warehouses and data marts as well as do ETL which feeds data into other applications. We test ETL by writing SQL queries against source tables and comparing results to those of the SQL queries against target tables. SQL queries for source tables are rather complex as they are based on a very complex set of business rules. Are you aware of any tools that can help us in this testing process?
|David Marco's Answer: If you are using Ascential DataStage, they have a data quality product calling Quality Manager. If you are using Informatica, I am less certain what their Quality Manager equivalent is. Regardless of the tool, all of the meta data that you are capturing from the SQL queries against the target tables needs to be stored in a meta data repository. A good repository is historical, so all of the data quality will be tracked over time. By storing this in your repository it will greatly aid in your testing.|
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access