Continue in 2 seconds

Intelligent Information Processing

  • July 01 2001, 1:00am EDT

The breakneck rate at which information is created, processed, packaged and communicated widely exceeds the rate at which individuals can absorb it, contributing to a constant struggle to transform data into actionable knowledge. One reason for this is that even though we live and work in an information age, our information is processed and managed in a manner that recalls the industrial age, especially in terms of an assembly line sequence of processing stages. Although significant advances have been made in the areas of distributed computation and parallel processing, most legacy information applications still operate in a linear fashion, creating "processed information" in isolated stages, the same way that cars are manufactured. This factory-style processing creates artificial barriers to the effective use of information, leading to lost opportunities and decreased competitiveness.

Alternatively, some organizations are moving toward a more knowledge-centric view where information is seen as a critical corporate asset that can be used to significant competitive and strategic advantage. This signals the migration from an industrial age view of data to a knowledge age view, where many data sets are aggregated, fused, enhanced and broadcast to share an enterprise-wide knowledge base. In this article, we will look at the barriers to exploiting the value of the corporate information asset and how to evolve from the data dark ages into a knowledge age organization.

The Information Product

Let's consider the result of an information processing activity as more than just completed work. Instead, consider the result of any information process (whether that refers to processed transactions, daily activity reports or publication of information to more complex analysis applications, etc.) as an "information product." Unlike other raw resources used in a manufacturing process, information is not "used up" at any particular stage of processing. This means that the data itself is available for multiple uses across a matrixed organization.

Unfortunately, vertical organizational boundaries and budget isolation issues prevent the cooperative efforts required to bear the costs associated with maintaining data's intrinsic value or exploiting its hidden value. As data sets are passed from one processing stage to another, the care needed to maintain the integrity of that data is ignored, leading to an erosion of the data's value. This results in decreased communication, increased costs, significant decreases in the quality of the information and ultimately a decreased ability to compete.

Corporate Information Asset

Instead of looking at data as raw input, we can view data like any other tangible asset that is managed as an enterprise resource. Information has an intrinsic value (e.g., a hot stock tip, a news item, a customer purchase record) which is enhanced through processing, packaging or presentation. By looking at all the information managed within an organization, we discover new ways to use the same bits of information to enhance many data sets across the organization.

For example, consider aggregating billing, sales and customer support databases to provide enhancements to a customer relationship management (CRM) system. A number of organizations have already discovered this and are working to strategically merge data from multiple sources. Unfortunately, in a stovepiped organization, some of the following issues doom attempts at information sharing:

The "Peter Principle" of Data Integrity: Dr. Laurence Peter's principle about management suggests that in an administrative hierarchy, people tend to be promoted to their level of incompetence. We paraphrase this principle with respect to data: The level of data quality rises as high as it needs to be within the organization that creates or uses that data. This implies that when other groups use that same data set, the data integrity probably won't live up to user expectations. This tends to devalue the corporate information asset.

The Petri-Dish Growth of Data Environments: In many organizations, the enterprise data environment grows out of different organizations the same way different kinds of bacteria grow in a petri dish. Each group has its own hardware, DBMS, software, etc., and these often don't reflect any kind of organizational standard. Because processing power and connectivity have increased, previously isolated groups now can interoperate, leading to issues associated with the lack of organizational data standards. For example, some groups may have data tables with a column for CUSTOMERACCT, while others use ACCTNUM, ACCOUNT, CUSTID, all referring to the same customer account number.

Personalization of Data Ownership: In companies that reward em- ployees based on performance, there is a tendency to allow (or even foster) internal rivalries as a motivation tool, promoting a powerful incentive to personalize data ownership. When this happens, employees tend to constrict the flow of the data they manage to any destination outside of their control as a means for improving their personal position within the organization. When individuals are able to claim ownership over a corporate information asset, it detracts from the ability to exploit that asset.

Weak or Missing Corporate Information Policies: A corporate information policy dictates how ownership, responsibility and dispute resolution are applied when managing corporate data. Most companies don't have a well-defined information policy. Those that do seldom enforce those policies, severely reducing the possibility of exploiting the information asset.

Organizational Hierarchies Remove Incentive to Improve: Boundaries of control are defined via a corporate hierarchy. Because groups typically focus on operating within short time frames, smaller groups concentrate resources on their internal tactical goals with little regard to issues associated with centralized assets. Therefore, there is little incentive to participate in any coordinated effort to improve anything outside the organization boundary.

Diffused Management Organ-ization Limits Power of Resources: Diffusing both control and resources limits a smaller group's ability to effect change. For example, the costs associated with data quality improvement may include a software license, maintenance, and personnel to analyze and manage a data quality program. While the benefits associated with such an investment may be spread throughout a company, the costs may exceed the budget allocated to any particular group; and the value added within the group may not justify the expense, even if other groups within the organization also benefit from the improvement.

Employee Turnover Leads to Knowledge Entropy: Employees are tasked with performing certain jobs, but every job has its idiosyncrasies ­ organizational behavior issues, business rules, corporate policies, etc. In addition, each worker tends to leave his or her own mark on the position, exploring efficiencies in performing the job. All of this can be viewed as corporate knowledge, although this knowledge is rarely, if ever, documented or transferred in any way other than "lore." As employees leave, they tend to take their knowledge with them, although by all rights this knowledge is also a part of the corporate information asset. In the presence of this kind of knowledge entropy, it is difficult to evolve into a knowledge- aware organization.

Employee Reward System is not Consistent with Corporate Goals: Frequently those employees who are in the best position to help exploit corporate knowledge are penalized when they do apply that knowledge. For example, consider the call center employee who is rewarded based on the number of incoming calls handled. When a customer calls to cancel an account, the agent treats this call as any other customer service call, trying to finish it quickly. Yet if that agent spends extra time to help the customer and uses collected customer information to retain the customer, the employee is penalized when the call quota is missed ­ even though the company benefits.


Savvy information professionals are recognizing the importance of high-value corporate information, and they are increasingly exploring ways to evolve into a knowledge age organization. Here are some suggestions to break down the barriers and start the evolution process:

Recognize and Centralize the Corporate Information Asset: The first step in this evolution is to recoganize the value of the body of corporate data and to explore ways to consolidate management of information. The ultimate responsibility for this lies with senior management ­ those who have been entrusted with the control and management of all other corporate assets. When information is viewed as an asset, its value can be exploited; without senior management support, this exercise will be futile.

Assess and Consolidate Enter-prise Reference Data: All organizations use reference data. For example, stock symbols, product codes, customer account numbers and state, country and currency codes are all examples of reference data. Unfortunately, because of the petri-dish growth phenomenon, group databases will have their own copies of the same reference data. This de facto replication leads to inconsistencies between data sets that should be standardized across an organization. If a reference data set is to be used across many groups within an organization, it makes sense to centralize the management of that reference data and provide access to that data to all participants. An assessment should reveal where the majority of the inconsistencies lie. A process of reference data consolidation not only provides a platform for enterprise-wide data consistency, but also centralizes responsibility for maintaining and improving the value of that data to the organization as a whole.

Provide Access to Enterprise Reference Data: Consolidating reference data will be meaningless unless the contributors to the centralized store have the same degree of access to the data going forward. Also, providing access to what becomes an organizational standard can only have a beneficial effect as more groups subscribe to the centralized data, not just by improving consistency, but also by standardizing the meta data language associated with the use of data. For example, all groups referring to customer account number using the same name is a good example of exploiting captured corporate knowledge.

Define a Data Ownership Policy and Means for Enforcement: A policy that dictates that the ultimate ownership of data resides with the corporation and how responsible parties must support data under their control is a major component of a knowledge age environment. At a minimum, a data ownership policy should enumerate:

  1. The senior managers supporting the enforcement of policy.
  2. The data sets covered by the policy.
  3. The ownership model (i.e., how is ownership allocated or assigned within the enterprise) for each data set.
  4. The roles associated with data ownership and the responsibilities of each role.
  5. A dispute resolution process.

Empower Data Management Roles to Augment Knowledge Management: If we are consolidating corporate information and defining a set of policies governing the use of that information, we are also obliged to define the framework in which that information is managed. A data ownership policy defines the roles associated with data ownership, such as data stewardship, data quality, access manager, etc. The people who take on these roles must be empowered by senior management to be able to enforce the defined policies.

Determine Value Attribution: Associated with any use of information is some aspect of value. Some information is of greater value than other information, depending on who is using it and how it is used. The responsibility associated with data ownership should be proportional to the value derived from that data. In turn, the budget contribution associated with the management and upkeep of data should also be reflected in an assessment based on value. Therefore, the next step in the evolution is understanding and documenting the value associated with information.

Let Policy Guide Knowledge Integrity: With an ownership policy and the budget assessment based on value, we now have a direct means for funding information maintenance. One of the most important aspects of information maintenance is ensuring data quality. When data sets are valued and managed as a corporate information asset, there will be clear responsibilities and enforcement processes for improving overall data quality.

Incent the Use of Corporate Knowledge: Modify the reward system to provide incentives for making use of corporate knowledge to improve profits. Identify those events where applying corporate knowledge will result in better business results, and reward employees when they do so. To follow our earlier example, when a customer calls to cancel an account, the call center agent should be rewarded if by exploring the customer's history, assessing that customer's lifetime value and providing some reasonable offer to address that customer's complaints, the agent manages to retain that customer.

Depersonalize Data Quality: If personalization of data ownership is a problem, the personalization of data quality responsibility is an even greater problem. To move toward a knowledge organization, remove any individual stigma associated with bad data by turning the data quality problem into a business process.

Define and Manage Data Quality Rules: The first step is in breaking the habit of associating poor data quality with anecdotes and stories. We all have expectations about high quality data, although frequently we are unable to articulate them. By making the effort to translate our hazy expectations into well-defined rules, we achieve two goals. We then have a framework for measuring how our data conforms to our expectations, and we can incorporate those expectations as part of the corporate knowledge.

Use the Rules: Defining and managing rules is one thing; using them is another. To operationalize data quality rules, use a framework in which rules can be integrated into an executable process. When rules have been translated into an executable form, they can be used to measure data quality against our expectations. Those measurements should be used to build a data quality scorecard.

Use a Data Quality Scorecard: A data quality scorecard is a tool used to manage the corporate information asset, providing precise methods to measure levels of data quality. When coupled with a data quality validation environment, we can evaluate the costs and impacts associated with low data quality. Ultimately, analyzing the flow information through an organization and assessing the economic impact enables the building of a return on investment (ROI) model to identify primary targets for information quality improvement.

Benchmark Against Well-Defined Goals

It is amazing to see how an organization is transformed once it has some control over its information expectations. For example, consider a company that uses supplied information products. Before defining and measuring conformance to data quality rules, that company's reliance on the quality of the data is subject to the supplier's perception of high quality. However, after measuring conformance to expectations, that company suddenly has some leverage for coercing the supplier to improve the quality (hence, the value) of the supplied data. Essentially, incorporating the definition of data quality rules, defining conformance levels and measuring conformance levels to those rules into the corporate information policy goes a long way in adding significant value to the corporate information asset.


Evolving into a knowledge age organization has significant business benefits:

Improved Enterprise Information Consistency: The consolidation of reference data and the subsequent mechanisms for providing access to that reference data leads to improved information consistency across the organization.

Elimination of Data Ownership Issues: Defined policies break the assumed ownership that occurs in companies.

Understanding of Information Flow: There is insight in understanding how information flows throughout an organization. By mapping the flow of data, a company can see how information is used, how it flows across the organization, who talks to whom, and the rates and priorities associated with different data sets. In turn, this provides meta-knowledge about data sets, data values and how those data sets are valued within the organization.

Creation of Opportunities: Dissolving some of the budget barriers that artificially grow within a company can enable strategic investments to be made in information and knowledge maintenance that were previously impossible. Viewing corporate information as a centrally owned asset provides a platform for long-term knowledge management programs that add directly to corporate profits.

The Bottom Line

Within the next five years, more organizations will recognize that managing the flood of information truly affects the bottom line. Organizations that are unable to control their use of information will be doomed to drown in this flood. Companies that see the strategic value of evolving into knowledge organizations will see direct benefits. This is a strategic evolution that may take a long time to effect, but it will result in improved profits and opportunity for significant competitive and strategic advantage.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access