Everyone’s talking about single version of the truth, but how often are our reports reviewed for accuracy? Several of our financial services clients demand that their BI reports are audited back to the source systems and that numbers are reconciled.
Unfortunately, this isn’t common practice across industries. When we work with new clients we ask about data reconciliation, but most of our new clients don’t have the methods or processes in place. It makes me wonder how engaged business users are in establishing audit and reconciliation rules for their BI capabilities.
No, data perfection isn’t practical. But we should be able to guard against lost data and protect our users from formulas and equations that change. All too often these issues are thrown into the “post development” bucket or relegated to User Acceptance. By then reports aren’t always corrected and data isn’t always fixed.
A robust development process should ensure that data accuracy should be established and measured throughout development. This means that design reviews are necessary before, during, and after development. Design reviews ensure that the data is continually being processed accurately. Many believe that it’s ten or more times more expensive to fix broken code (or data) after development than it is during development. And, as we’ve all seen, often the data doesn’t get fixed at all.
When you’re building a report or delivering data, ask two questions: 1) whether the numbers reflect business expectations, and 2) if they reconcile back to their system of origin. Design review processes should be instituted (or, in many cases, re-instituted) to ensure functional accuracy long before the user every sees the data on her desktop.
Evan Levy also blogs at evanjlevy.com.
The data supports business in their requirements for information when appropriate context is added, however regulators and the like monitor the things they know about and don't easily recognise changes in behaviour unless they are watching for them.
When behaviour changes slowly over time, the data may be accurate but the interpretation is flawed.
Terms like; "single version of the truth" is a great marketing tag line. We have solved one of the conundrums of the universe and found the single source of all truth!
The challenge with BI data is that in order to validate what is published every aspect of the data has to be examined. The source, the transformations, the model used to manipulate the data and the assumptions and bias's made by the people who created the published data. This detailed work will casue delays in publsihing the data and so few perform this due diligence. We have come to believe that because the data has been processed by a system it is inherently correct.
If we examine the financial crisis from a strictly data perspective we will find that the data was all there and manipulated to reflect the expectations of the target audience. This is truly a "single version of the truth". The data showed everyone making money. Unfortunately the data was correct but the interpretations were wrong. How do we ensure correct interpretations? This is not just a regulatory problem.