If the early indication holds true, the knowledge management revolution will not be led by the IT organization. The knowledge management revolution can only be let by those who can and do manage knowledge. Those who do not learn from their mistakes cannot teach the rest of the enterprise to do so. The fact of the matter is that 62 percent of information professionals believe that their organization and others have not learned important lessons from their Y2K experiences and have not changed their data design processes.

In my March column, I conducted a survey to see what we have learned from the enormous expenditures organizations made to "fix" the Y2K data design defect. According to the responses, apparently not much. The first question asked was, "Which general conclusion about Y2K will prevail?" I asked readers to indicate which of the following statements reflected their Y2K experiences:

A. Our organization rose to the occasion, solved the problem, and now it is time to get back to business as usual. We will keep on developing applications and databases as we did before.

B. We could have prevented the Y2K problem if we had designed our databases properly. We will improve our data design processes [to prevent design defects, not just dates] as a result of this experience.

The second questions was, "If positive change is going to happen to prevent recurrence of these kinds of problems, who will lead the way: information management professionals, application developers, business professionals or external entities (such as consumers or regulatory advocates)?"

The answers surprised me. They might also surprise you. Download a graph and discussion of those results as well as a graph of the first question from www.information-quality.com. Select "Information Quality Resources," then "Newsletters and Articles."

Poorly designed databases will, at some point, force the organization to redesign (rework) and implement corrective actions or scrap the entire legacy and replace it (with not-so- inexpensive enterprise-wide systems). The real casualty in this, however, is suboptimized business performance. Poor data design and accessibility prevents the enterprise from capturing and exploiting knowledge. Without complete, accurate and accessible shared knowledge, we cannot fully understand our customers and the effectiveness of our products from our customers' perspective, nor can we "think our customers ahead" to anticipate their needs for tomorrow's products.

Some of the 208 responses (coming from the readers of this column, Y2K presentation attendees of the March Meta Data/DAMA Conference and e-mail respondents) are telling in more ways than one:

  1. The Y2K remedial work was "just seen as a cost of doing business."
  2. Most companies did "rise" to the occasion and fixed "bad design" and lack of foresight at an enormous expense. Unfortunately most companies haven't learned from the harsh Y2K lesson and are still developing applications and databases with the same design and lack of foresight.
  3. IQ and DA people are upset about this attitude but have not made a sufficient case.
  4. No "visionary" learning took place about how we develop applications.
  5. I am not getting the perception from our developers or our management that there were any design or development flaws, or anything that could have been done to prevent Y2K, so it's business as usual.
  6. I believe most organizations and individuals, especially those "data specialists" (modelers and DBAs), will recognize the truth of the second answer and privately acknowledge it to be the correct one. This and various statements acknowledging the importance of data quality make it tempting to predict a true reform in behavior. However, I believe that when the memory of the pain recedes, most people and organizations will return to the old "we don't have time to do it right" scenario.
  7. I wish the answer was B (we will improve our data design processes), but I'm afraid A (keep on developing as before) will prevail. It's much more fun to be a hero than admit we didn't design correctly and change.

Those IT organizations which celebrated victory in their Y2K corrective actions unfortunately celebrated the wrong thing. They celebrated the fact that they had spent money creating a major problem and then turned around and spent more money fixing it! To be honest, many good people spent much time and overtime helping to prevent what would have been a catastrophe had it not been addressed. This is not to take away from their hard work.

My experience with organizations around the world is that information scrap and rework, whether the data itself or data design defects, are simply accepted as normal costs of doing business (response 1). This is the same misperception that was held in manufacturing before the first quality revolution. The second quality revolution will transform information processes when information management and information quality professionals demonstrate to management that these costs are not normal and implement process improvements to prove it.

Many responses indicated a lack of empowerment and resignation (represented in response 6). The first habit from Stephen Covey's book, The 7 Habits of Highly Effective People, is "Be proactive."1 Those in the organization responsible for information management and data design must be proactive to change how data design is accomplished.

Information professionals must quantify the costs of nonquality data design in business terms. In the same way manufacturing quantifies its costs of nonquality, so must information managers. One respondent correctly identified how information management professionals can exploit the Y2K problem and turn it to advantage. "While cost figures are still available, we should use it to do a root cause analysis. [The problem is] probably traceable to data design flaw. If so, DA [data administration] must use to illustrate the importance of data design." These techniques and processes are described in Chapter 7 of my book, Improving Warehouse and Business Information Quality.

If something is worth doing, it is worth doing right. Data design is not a support process. It is a process of building the knowledge infrastructure of the enterprise. Incorrectly designed databases lead to dysfunctional learning organizations. The argument, "we do not have time," is, first of all, a red herring. After all, we had time to redo the data design for Y2K. Secondly, it is a false assumption to say speed and quality data architecture are mutually exclusive. You can have quality data design and speed of delivery at the same time. It does take the investment of bringing the right business subject-matter experts together and rapidly facilitating a common shareable subject model. More than 20 checklists for data definition and data model quality are described in Chapter 5 (Assessing Data Definition and Information Architecture Quality) of Improving Warehouse and Business Information Quality.

Who wants to admit problems when you can be a hero. First and foremost, the information quality environment is a nonblame, nonjudgmental environment. If we have defects, we do not have defective people, only defective processes. There is a difference in admitting to problems and denying them. If we deny there are data design problems, we fail to be good stewards of our enterprises' second most important resource. The data design process can be improved. See Chapter 9 for how to use the Plan- Do-Check-Act Shewhart cycle to improve data design processes.

One person who answered, "We will improve our data design processes," also cited a potential pitfall. "Since we diverted precious design and development resources and time fixing Y2K problems, we need quick solutions to catch up with the changed business environment. Data design may well be one of the casualties of expediency ... once again." I am reminded of a comment Deming made in describing his first point of quality: "Create constancy of purpose toward improvement of product and service with the aim to become competitive and to stay in business, and to provide jobs."2 Deming states that management has two sets of problems, those of today and those of tomorrow. It is easy to become tangled up in today's problems, becoming ever more efficient in them. Putting out the fire of the Y2K problem (the problems of today) left organizations behind in that value work (solving the problems of tomorrow). Will organizations, forced to delay their value work by the Y2K scrap and rework effort, try to catch up by shortchanging the planning and design of tomorrow's knowledge bases? Without a plan for the future, the enterprise cannot stay in business. If an organization cannot learn that firefighting is not quality improvement, how can it survive?

Please let me and your fellow information professionals know how you have improved your data design processes at www.information quality.com on the IQ Forum under the Information Quality Resources button (in the Data Definition and Information Architecture thread).


1 Covey, Stephen. The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change. New York. Simon & Schuster. 1989.

2 Deming, W. Edwards. Out of the Crisis. Cambridge, Massachusetts: MIT Center for Advanced Engineering Study. 1986. P. 23.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access