Sid Adelman's Answer: The reconciliation is critical. If you do not reconcile, you have no way of knowing if the ETL process completed successfully and the business users rarely have any way of performing their own reconciliation. This means they may be dealing with old data (they may assume it is current) and the data may be incomplete. These users will produce incorrect results and when the cause is determined, they will point to you. You should be looking at tie-outs which include certification of the number of rows loaded and verification of numbers within the database. Tie-outs are standard features of the ETL tools but they have to be turned on and someone has to verify the results and this must be done each time the ETL process is complete.
Sid Adelman is a principal in Sid Adelman & Associates, an organization specializing in planning and implementing data warehouses, in data warehouse and BI assessments, and in establishing effective data architectures and strategies. He is a regular speaker at DW conferences. Adelman chairs the "Ask the Experts" column on www.dmreview.com. He is a frequent contributor to journals that focus on data warehousing. He co-authored Data Warehouse Project Management and is the principal author on Impossible Data Warehouse Situations with Solutions from the Experts and Data Strategy. He can be reached at (818) 783-9634 or visit his Web site at www.sidadelman.com.
Be the first to comment on this post using the section below.