Issues and Opportunities in Data Quality Management Coordination
Information Management Magazine, April 2004
There is a growing cross-industry recognition of the value of high-quality data. Numerous reports issued over the past two and a half years indicate that information quality is rapidly increasing in visibility and importance among senior managers. According to the 2001 PricewaterhouseCoopers Global Data Management Survey, 75 percent of the senior executives reported significant problems as a result of defective data. In 2002, The Data Warehousing Institute published its Report on Data Quality in which they estimated that the cost of poor data quality to U.S. businesses exceeds $600 billion each year.
Not only that, two critical pieces of legislation passed within the past few years impose strict information quality requirements on both public corporations in the U.S. (the Sarbanes-Oxley Act of 2002) as well as U.S. federal agencies (The Data Quality Act of 2001). Both of these laws require organizations to provide auditable details as to the levels of their information quality.
Some Interesting Issues
Advertisement
The emergence of the value of high-quality information coupled with these new regulations has prompted many organizations to consider instituting a data quality management program, either as a separate function within a logical line of business or at the enterprise level. While this is admirable, there are a number of relevant issues that can impede the integration of information quality concepts. Some these critical issues include:
Questions Regarding Data Ownership: Unless there is a set of clearly defined data ownership and stewardship policies, there are bound to be some questions regarding responsibility, accountability and authority associated with auditing and reviewing the quality of data sets.
Application-Based Data Management: The systems and applications that comprise an enterprise environment may be structured in way that the business manager for each system has authority over the information used within that system. Consequently, each application in isolation has its own requirements for data quality. However, as we move toward a more integrated application environment as well as explore the development of an enterprise architecture, it is possible that application data may be used in ways never intended by the original implementers. This, in turn, may introduce new data quality requirements that may be more stringent than the original, yet there may be hesitation by the application teams to invest resources in addressing issues not relevant within their specific applications.
Administrative Authority: In some instances, the information used in an application originates from a source that is outside of the application manager's administrative authority. For example, in an application that is used to aggregate information from many information partners, many of the data quality issues are associated with problems at the partner level, not within the aggregated system. Because the problems occur outside of the centralized administrative authority, even if the data is modified/corrected at the centralized repository, it does not guarantee that the next submission would not still include instances of the same problems.
Data Quality in an Advisory Role: In application-oriented organizations, another impediment to data quality coordination relates to how one deals with improving the quality of specific data used within an application when that data is sourced from an external data supplier, and is consequently managed outside of the application manager's jurisdiction. Although in some organizations the project structures may already have an associated data qualtiy function, the more important issue is whether in practice all participants will cooperate with the data quality improvement process.
Data Quality as a Business Problem: In many organizations, business clients assume that any noncompliance with expectations results from data quality issues and needs to be addressed by the technical teams. However, in reality, the business rules with which the data appears to be noncompliant are associated with the running of the business. Consequently, those rules should be owned and managed by the business client as opposed to the technical team members, whose subject-matter expertise is less likely to be appropriate to address the problems.
Impact Analysis: Anecdotal evidence may frequently inspire attitudes about requirements for data quality; however, in the absence of a true understanding of the kinds of problems that take place, the scope of the problem and the impacts associated with the problems, it is difficult to determine the proper approach to fixing the problem as well as eventually measuring improvement.
Reactive versus Proactive Data Quality: Most data quality programs are designed to react to data quality events instead of determining how to prevent problems from occurring in the first place. A mature data quality program determines where the risks are, the objective metrics for determining levels and impact of data quality compliance, and approaches to ensure high levels of quality.
Data Quality in an Advisory Role
In essence, many of these issues stem from the simple fact that the persons entrusted with ensuring or managing the quality of data usually do not have authority to take the appropriate steps to directly improve data quality. Instead, the data quality management function may exist to understand and coordinate the data quality activities (reactive or proactive) that currently exist within an environment as well as work toward developing a mature data quality capability within the enterprise. However, the management coordinator role is likely to be an advisor to the application system data "owners." This advisor is tasked with inspiring those owners to take responsibility for ensuring the quality of the data.
Opportunities at the Advisory Level
To summarize, most issues derive from the fact that a large part of data quality management, especially at the enterprise level, is advisory. To add complexity, there is an expectation that as soon as data quality professionals are brought into an organization, there should be some visible improvement to the data. This poses quite a quandary at times because the data quality manager is viewed as having responsibility for some action without necessarily having the authority to make it happen. The key to success, as we have learned with a number of our clients, is to exploit the advisory role and use internal procedures to attach the responsibility to the already existing information management authority. In other words, we guide those information managers in the data ownership positions to accept the responsibility through a combination of the advisory role and the organizational system development life cycle policies, standards and procedures to which those information managers are bound.
Page 1 of 3.
