Related Links

Web Seminars

A colleague of mine recently told me that he is working for a consulting firm on an information quality (IQ) project with a client where they were just "playing with data quality," not really doing it. As he shared some details, we both agreed it was a sad state of affairs. This article explains what is required to really do information quality and describes a methodology for how to do it.

It would be really nice if we could solve IQ problems by implementing a one-time project. However, IQ problems cannot be solved that way because IQ problems are caused by broken, out-of-control processes, which are influenced by industrial-age, ineffective management systems that reward the wrong things.

It would also be nice if we could solve IQ problems by simply dropping IQ software systems into place to completely clean the defective data. Alas, IQ problems cannot be solved that way either. Software cannot completely remediate the limitations and weaknesses of broken processes outside the software system, nor can it completely eliminate the factors introduced by the human element in information production. IQ problems cannot be solved permanently unless you address the organization's management systems that set performance measures and influence employee behavior.

There is no panacea or silver bullet for Total Information Quality Management (TIQM), just as there was no panacea or silver bullet for manufacturing quality management. TIQM requires a sound, defined set of processes ­ implemented and executed with discipline.

Importance of Methodology

Five Reasons to Implement TIQM

Every process that produces quality results consistently requires the process to be defined (so that it is repeatable), controlled (so that it is consistent) and improved (so that it eliminates the causes of defects or error). This is true whether the process is manufacturing a car, filing a tax return, taking a catalog phone order or processing an insurance claim. This is also true whether the process is developing an application system, developing an enterprise data model, conducting an IQ assessment or facilitating an information process improvement.

1. TIQM was developed using proven quality management principles, methods and techniques successfully applied to manufacturing and other industries.

When I began my work in IQ management, Tom Redman had not yet written his first book (published in 1992). There was no formal program of IQ at any university. I became interested in information quality by discovering W. Edwards Deming ­ first through Mary Walton's book, The Deming Management Method. From that, I moved to Deming's Out of the Crisis in which he outlines his famous 14 Points of Quality.

Deming's 14 Points transformed my professional focus and consulting practice. I recognized immediately that quality management principles are not ancillary to information; they are directly applicable to data as the product of business processes in the same way an automobile is the product of manufacturing processes. Then I read Masaaki Imai's Kaizen: The Key to Japan's Competitive Success. It confirmed what I learned from Deming: Quality management is a continuous improvement of processes to eliminate the causes of defects. Another major influence was Philip Crosby's Quality Is Free: The Art of Making Quality Certain. It confirmed again that the business case for information quality is the same as the business case for quality management in general ­ measure and eliminate the costs of scrap and rework. I also learned from Crosby the five distinct stages of organizational maturity in the journey and transformation to an effective culture of quality management and business performance excellence.

In addition to these quality gurus, I studied Joseph Juran (Juran on Planning for Quality), Kaoru Ishikawa (Guide to Quality Control), Yoji Akao (Quality Function Deployment: Integrating Customer Requirements into Product Design) and the Baldrige National Quality Program's Criteria for Performance Excellence. Now I am studying Six Sigma and have mapped TIQM's process steps into Six Sigma's define-measure- analyze-improve-control (DMAIC) methodology.

TIQM was not developed by looking at other data quality or information quality methods and techniques, but by studying quality management principles, techniques and processes from the leaders of the quality management revolution, understanding them and then applying them to information.

If you doubt the pertinence of manufacturing quality management principles to information, please note that before Deming went to Japan and taught the Japanese his quality management principles, he applied them at the U.S. Census Bureau in the 1940 national census process ­ a pure information quality application!

2. TIQM is IQ software tool open.

TIQM is not biased to any specific software tools. Rather, it provides a framework for exploiting IQ software as well as other quality tools and techniques. It classifies the capabilities of IQ software and where they fit in the TIQM process methodology. Tool categories include: assessment; rule discovery and analysis; reengineering, cleansing and transformation; enhancement; defect prevention; and quality control and management.

You may ask, "Why should our company implement TIQM? After all, we have implemented IQ software/our own methodology."

First, if you already have an IQ methodology, compare it to TIQM and augment it if necessary. Remember that every process is a process that can be improved (Deming's Point 14) ­ even IQ processes.

Many of you may have excellent IQ software in which the supplier has provided a methodology. Do not throw that methodology away, but examine it against TIQM to see if there are gaps. Before I began IQ consulting, I noted that when third-party software suppliers developed methodologies to augment their products, those methodologies were biased based on the features of each supplier's products. Important methodological steps were missing if the product did not provide a function to support that step. The same omissions can happen in IQ software providers' methodologies. For example: IQ assessment software suppliers may provide a methodology for IQ assessment that includes steps to define, measure and report completeness and validity assessments. Does the methodology have a step to measure accuracy by physically comparing data to the real-world object (or a recording or observation of an event)? Accuracy assessment, one of the most important IQ characteristics for knowledge-workers and end customers, cannot be conducted electronically. Comparing data to a reference database, such as a postal database, does not represent a measure of accuracy; it represents a measure of consistency. In order to interpret the result of this comparison, you must know the accuracy of the reference data. For example, an address may be determined to be a valid address, but the person believed to live there has long since moved.