Customer Data Disorder: Part 1 - Defining Customer Data Disorder

Published
  • September 22 2008, 11:38am EDT

The goal of IT is to deliver accurate, complete and relevant information in a secure fashion to people and processes on demand. Information about the parties you do business with is a critical asset to business. As business organizations have grown organically and through acquisition, data about customers is stored in many places in the enterprise. Each data store is defined differently, used by different business processes and updated by different business applications. The keys for and links between the data that describes customers gets out of alignment with the characteristics of customers in the real world. Customers change frequently, as do the business processes that manage customer information, the business logic in applications and the metadata associated with the data stores. The difference between people and legal entities in the real world and the information we have about them is called customer data disorder. When the condition is advanced, business performance suffers.

Complete and accurate customer data is critical to business. Information about parties (people and legal entities) that large enterprises do business with is frequently incomplete and inaccurate. Questions like, “How many customers do we have?” are often difficult to answer. Poor quality party data negatively impacts business performance across functions. It masks a true understanding of risk and profitability. It impacts the effectiveness of marketing and sales. It hampers the productivity of service. It results in conflicting reports and makes it more difficult to comply with legal requirements and regulations. “Improving data quality/information integrity” was the most pervasive technology concern cited by 58 percent of the 653 respondents to a recent CSC/FEI/FERF survey.1

“Parties” are the people and legal entities you do business with. Parties are people and legal entities that enter into contracts for goods and services and include person parties and business parties. Parties are often referred to by the type of relationship they have to a business account, such as owner of a financial account, subscriber to a telecommunication service or member of a health care plan. Parties also have important relationships to other parties in three relationship types: person to person, person to business and business to business. Examples are individuals in a household, contacts in a business organization, doctors in a medical practice and establishments in a legal entity.

Customer data disorder is the conflict between the characteristics of parties in the real world and the information you have about them. In IT terms, there is the entity in the real world and the information about the entity in the world stored in databases and managed by business applications. Party is the entity itself in the real world. Party data is the information entity that contains information about the party entity in the real world. According to IDC, the average company has 49 applications that operate on 14 different customer databases and, on average, no more than 20 percent of customer data resides in a single location.2 Very often there is a different meaning for party within a business based on function every place there’s party data. This includes different business definitions, different logical models, different application logic, different data values and keys that cannot be easily reconciled across sources. Additionally, people and businesses in the real world change constantly. And of course, the speed of business and the number of contact channels used continues to accelerate and expand. No wonder “single view of customer” continues to come out on top in various surveys related to information management, business requirements and challenges. Customer data disorder is the serious business condition that arises from a deep misalignment between the parties you do business with and the information you have about them across the enterprise.

Party Information Comes in Layers

An important nuance when working with party data is to understand the difference between the data that attributes the party itself and the data that describes the party in the context of a business relationship. Customer data disorder is so insidious in part because there are many legitimate contexts within a single business organization that differ in function yet depend on core party data that must be consistent and accurate across sources. Party identifying attributes are the facts that characterize a single party. Naturally, significant differences exist between person parties and business parties, and location plays an important role in both cases.

While there is one party - like a person - in the real world, there are many instances of the information entity or set of identifying attributes in the various databases that hold business information in an organization. This condition is commonly referred to as duplication and occurs within a single data source and also across sources. Party keys link different sets of data instances that describe the same party in the real world. The process referred to as deduplication removes the duplicates in a single source by first determining that two or more sets of data attributes describe the same party in the real world. A new group key is then assigned to all the instances of those identifying attributes. This is often referred to as a cross reference between the source record keys and the group keys that are assigned to record sets of identifying attributes that describe the same entity in the real world. Duplicates are eliminated because the group keys more accurately represent the actual number of parties contained in a data set. The process works the same way across data sets. In both cases, the duplication percentage reflects the amount of redundancy - or how inaccurate the party data really is. The higher the duplication percentage, the poorer the quality of the party information.

Party relationships are combinations of party keys that describe a specific relationship between two parties. Spouse and household are examples of person party to person party relationships. Relationships between business parties often describe the party affiliation in the context of a buying or distribution relationship or the legal relationship between business entities. Contact and membership groups, professional affiliations and employee to employer relationships are examples of person party to business party relationships. Poor quality of party keys and relationships is a root cause of customer data disorder.

The first two party information layers, the core party data, describe the person or business party itself (see Figure 3). In data quality terms, the attributes, keys and relationships can be said to be free from defects. The attribute values are or are not correct, with an explicit number of defects, at a point in time. The set of keys has a known level of duplication. On the other hand, the next three layers are also judged subjectively as to their fit for use for a particular business purpose. Because a business may interact with many parties playing different roles, the profile, transactional and analytical information may differ significantly based on business function and customer party type. Extended party profile information often comes from external data sources like Dun & Bradstreet for standard industrial classification codes, revenue and employees at a business party location as well as Acxiom for person party demographics, such as baby boomer or generation X. Party transactions catalog specific interactions like contacts, marketing offers, purchase history and service calls. Transactions usually have their own unique identifiers and are often associated with an account identifier that in turn is linked to party identifiers. Party analytics is high-value information derived from the underlying party information layers, like customer profitability, and is particularly dependent on resolving party semantics across sources.

Business Context for Party Information

Customer data disorder is a function of the complete ecosystem where the party information layers are managed and consumed. In information management terms, the data about the information entity is called metadata. Yes, there is the entity itself (the party in the real world), the information entity (the data maintained about the party) and the metadata about the information entity (the data about how the information entity is defined and organized). The metadata about the party information entity is often described as a logical model in a database design tool. That is where you would find simple things like the names of fields to store data values about party identification attributes. That is also where permissible relationships between parties are described, as well as the relationship of a party to a business account. Oracle’s Trading Community Architecture and SAP’s Business Partner are examples of widely installed models that represent the complexity of party information in business context. These models get implemented physically in a database management system, where the data associated with party is actively managed (created, read, updated and deleted). The business logic in the application system works directly with the database management system. The business process or activity system uses the business application and has final authority over business rules and process steps that consume and manage the party information entity. The metadata, logical model, physical database, application logic and business process all provide the context to manage party information.

Party Information Variance in a Single Business Context

Customer data disorder means there is significant variance between information about the customer party in the information entity and the party itself in the real world. It’s a failure to identify, map and manage four kinds of changes. First, people and legal entities change frequently in the real world, and business has little control over the entity itself. This is dramatically different than other managed business entities - like product - where the business organization is responsible for the entity itself as well as the information entity. Second, the context to manage the information entity also changes frequently. Changes to business process, business rules and application logic all impact the information entity. Third, the metadata about the information entity, both logical and physical, tends to fall behind changes in the business context. Finally, the information entity itself changes, often at the mercy of many different business and data integration applications. Customer data disorder arises when these changes are not managed sufficiently to maintain an alignment in the accuracy and completeness of the party information entity as it describes the customer in the real world.

A Great Opportunity to Improve Business Performance

Now that we have a clear definition of the condition, part two of this series will explore the outward signs and symptoms as well as the tests and methods available to reach a firm diagnosis in a specific business context. The third part in the series, treating customer data disorder will outline options to remediate a specific party data ecosystem. Perhaps a total cure is out of reach, but any sustained progress that produces better party information will improve the performance of your business.

References:

  1. Computer Sciences Corporation (CSC), Financial Executives International (FEI) and the Financial Executives Research Foundation (FERF). “Ninth Annual Assessment of the Information Technology (IT) Practices, Priorities and Problems that Confront Today’s Senior Financial Leaders,” July 2007.
  2. John F. Gantz. “The Expanding Digital Universe: A Forecast of Worldwide Information Growth through 2010,” IDC, March 2007.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access