In today's competitive business environment where real-time information must be met with real-time response, organizational data quality is a critical concern. Customer loyalty can be determined by the ability of a business to communicate effectively and accurately with its customers and prospects. Operational efficiency also depends on the availability of reliable and timely data so that forecasting, planning and other essential functions can be accurately performed. It should be no surprise then that enterprise data quality is one of the most often cited concerns among IT executives.

The explosive growth of customer relationship management (CRM), supply chain management (SCM) and enterprise resource planning (ERP) applications is well known. The promise of these far-reaching systems lies in the understanding, control and optimization of business processes. However, as with any system that relies on the analysis of enterprise-wide data, the effectiveness of CRM, SCM and ERP is dictated by the quality of information being used to drive them. Do you want to proactively manage your order tracking or aggregate customer product purchase history? Without ensuring the quality of your data, these processes could be rendered completely ineffective.

For example, SCM is essential to any business that relies on upstream suppliers. Unfortunately, keeping track of a number of vendors is a daunting task ­ and is a problem with which most operational executives are all too familiar.

A single company might appear in a corporate database under numerous spellings, addresses, account numbers, contracts and contact listings. In addition, parent- child relationships among vendors and their subsidiaries or branch offices add to the complexity. With better data, companies could garner a better understanding of their suppliers and not only streamline their purchasing and accounts-payable procedures, but more easily manage relationships with vendors. Better data quality could also enable businesses to optimize forecasting, inventory and distribution.

Of course, no discussion of data quality would be complete without addressing the e-commerce arena. The real-time nature of the medium together with the high sales volumes generated through e-business channels make it an operational necessity that poor data quality does not slow down or otherwise interrupt such a critical revenue stream. It is well recognized that e-businesses face unique challenges in trying to create lasting relationships with customers who are interacting with them in a less personal environment. Because of this, many of these companies are beginning to recognize the significant threat to customer goodwill that poor service, such as missed delivery of goods or improper personalization, might bring about. It is of vital importance that businesses create lasting customer relationships with online buyers. Thus, e-businesses must be able to promote, market and deliver goods in an accurate and timely manner.

Successfully fulfilling online orders also depends on having good data quality. Much to the dismay of the e- merchants, customers are now their data entry clerks. Even for more traditional businesses, such as professionally staffed call centers, typical data entry error rates range from three to five percent. With online customers now performing this role, data entry errors such as addresses are substantial ­ especially considering that a high proportion of online shoppers order products for friends and relatives at addresses different from their own. Without the ability to check the accuracy of customer information while the customer is still on the site, a merchant is taking a tremendous risk in assuming the customer information that has been entered is correct. By verifying the accuracy of a customer address, e-businesses can save substantial sums not only by avoiding returned delivery charges, but also by maintaining customer goodwill and ensuring repeat business.

Organizations, particularly e- businesses, must also possess the ability to verify data from international customers. According to Forrester Research, worldwide e-commerce will hit $6.8 trillion in 2004, with only half of this figure coming from the United States. So e-businesses must also have data quality systems that can manage this tremendous upsurge in global commerce.

Overview of Processes

Enterprise data quality is the ability to provide data quality practices to customer data from the point of inception and throughout the customer relationship. To ensure effective enterprise data quality, organizations must cover four bases.

First and foremost, most data quality initiatives begin with the cleanup of the legacy data that an organization has collected. This normally is done locally on the system where the data is housed. By beginning with this step, enterprises can ensure that the data in the operational system has been cleansed and standardized. Thereby, functions that rely on operational (legacy) systems, such as the printing of invoices or statements, can utilize accurate, up-to-date customer information while data quality practices are initiated.

Next, organizations must implement real-time, front-end data quality systems to assure that data quality is optimized at the point of entry into the enterprise. This is true for call centers, Web sites, point-of-sale systems and all customer touchpoints.

Organizations must also possess the ability to do ongoing asynchronous (sometimes called batch) cleanup to address the inevitable degradation of data quality that occurs over time. This will, for example, keep track of address changes for customers who have moved. Although no system can ever be one hundred percent effective, utilizing batch cleanup in conjunction with real- time data quality solutions ensures that the highest possible level of data quality is maintained.

Finally, the only way that problems associated with data quality can be effectively recognized and addressed is if organizations monitor, analyze and report upon these processes within the enterprise. Ultimately, it is people, not machines, that run organizations. Users' interactions with technology must be addressed. The need for organizational decision-makers to monitor the enterprise and determine where problems exist within procedure or management issues is a critical component in utilizing automated systems. For instance, if an organization has automated systems that indicate that the average number of errors from its Kansas City call center is fifty percent higher than that of its Denver facility, this might call for increased training or more effective management.

Technological Considerations

Getting to the Data. Prior to implementing any comprehensive data quality initiatives, the data (which often resides in numerous application systems and platforms across the enterprise) must be made accessible to the data quality verification, cleansing and enrichment systems. The distributed nature and number of disparate platforms across the enterprise make accessing the data a particularly challenging task. Enterprise application integration (EAI) is the process whereby key elements from each data repository are removed, sent through a data quality process and then returned to that repository. Using EAI does not entail building a data warehouse ­ in fact, most companies still do not have one. With EAI, businesses have a comprehensive system for accessing the information, even if it resides in multiple databases. The same process must be applied if it is being done independently to multiple systems.

If a company wants to build a data warehouse, a more traditional approach will suffice. Extract, transform and load (ETL) gives organizations the ability to extract data from multiple sources and compile the information in a central repository, typically done through scheduled batch processes. Data quality practices are normally used during the transform phase of ETL. To truly adhere to the enterprise data quality mantra, companies would still need to find a solution to put clean data back in the source.

Real-Time Systems. The real-time nature of today's business systems requires the implementation of real-time data quality solutions. This might mean specialized software packages that are designed for high-speed transactional processing or, more frequently, online systems that can be accessed remotely, such as ASP offerings. Real-time systems can provide front-end cleansing and validation and/or data enrichment for Web sites, call centers and other data entry points. Within the last year, online offerings have been introduced by a number of vendors and are being met with widespread acceptance by Internet retailers.

Asynchronous/Batch Systems. For many companies that have substantial customer records, and even for those with modest customer databases, regular database cleansing is essential for maintaining customer relationships. Many of these applications are probably well known to the reader. In fact, most date back twenty years or more and have their origins in list management applications. These applications can be run routinely on the source platform or through an asynchronous process to a centralized server.

Reporting. As mentioned previously, technology and automated systems are not enough. Unless those systems are able to monitor, analyze and report on the data quality issues facing the enterprise, proactive steps cannot be taken to improve the execution of the overall process. Identifying those systems or areas of the organization where error rates are higher or lower than average can provide CIOs and other executives with the information they need to make any requisite procedural or managerial changes.

In conclusion, comprehensive implementation of data quality systems must provide critical feedback to not only lead to more effective data quality processes, but to be utilized for improved performance on an organizational basis.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access