In his book Chaos, James Gleick described the "Butterfly Effect."1 Thirty-five years ago, scientists began realizing how the single flap of a butterfly's wing in China has the potential to start a chain of events in the atmosphere that results in torrential rain in the United States. The "Butterfly Effect" was given a technical name, "sensitive dependence on initial conditions." He pointed out that for centuries, people have been aware of how little things can escalate into big problems, as illustrated by this children's nursery rhyme. For want of a nail, the shoe was lost;
For want of a shoe, the horse was lost;
For want of a horse, the rider was lost;
For want of a rider, the battle was lost;
For want of a battle, the kingdom was lost!
I wonder how the rhyme would go if it started:
For want of a comment, the business rule was lost...
The purpose of this article is to create a definition for legacy systems, the intent being to avoid building new legacy systems in the future. We do want to avoid it, don't we? We will find that at the heart of this definition lies the escalating problem of "uh - oh - yikes - what - do - i - do - now - i - hope - they - cancel - the - project - client/server - is - too - complicated - anyway." Or put another way, at the heart of this definition is the identification of sensitive dependence on initial conditions: documentation.
Would you be willing to take an information technology (IT) equivalent to the Hippocratic oath: primum non nocere?2 Professional people have taken this code of ethics since Aristotelian times, some 2,500 years ago. In English it means, "Above all, not knowingly to do harm." Would you take this oath as an IT professional? Would your colleagues?
If a doctor gave you a prescription for medicine, but was too busy to include the instructions on how often and in what quantities you should take the medicine would you complain, maybe even demand the information? Why would we deliver systems and not provide adequate documentation? If we were forced to take an oath or pledge, we probably wouldn't have such a tough time meeting ISO 9000 quality standards or moving up the capability maturity model (CMM) scale, which was originally developed by the Carnegie-Mellon Software Engineering Institute.
The ISO 9000 standard is made up of four parts: 9001 is for service industries, 9002 is for manufacturing industries, 9003 is for computer hardware and software and 9004 is for quality management. In essence, their criteria for quality mean that you should "plan your work and work your plan," with the extra caveat of documenting everything you plan and doing everything you document.
CMM has five levels of maturity. Starting with the initial level, level 1, it classifies organizations whose software development or maintenance process is chaotic in nature. Symptoms include a development environment that is not stable. Planning is ineffective and not managed. Management commitment is driven by reaction. Is this starting to sound familiar? This can create a crisis and, in turn, what was left of the plan is abandoned. Level 2 introduces basic management control where software processes are repeatable. Project planning and tracking are seen as necessities and not as necessary evils. This is the first stage of quality. Level 3 suggests that an organization's process is adequately defined, and the pain of change management has eased. Unfortunately, this is where some organizations plateau and stop growing. Level 4 dictates that the software process is well managed, and level 5 is the optimized state of process control.
To reach level 5, your organization must have a three- or four-pronged architecture: information/business, application and technology. Processes must be well documented, accessible and repeatable. Services and resources must be either reusable or sharable. The organization must be capable of turning around delivered solutions every 90 days. Regardless of complexity, this is attainable if you have control of the initial condition of sensitive dependence. Like erecting a sky scraper, if the footings are inappropriate, it won't matter how elegant or ornate the marble entryway is. You wouldn't see it because it would be covered by the rubble of 50 floors.
Formal detailed documentation has long been the Achilles heel of many IT organizations. As far as maintaining (keeping up to date and accurate) the documentation, forget it. That's just a pipe dream. Though, in my heart of hearts, I do believe that some of you reading this have organizations in a state of documentation nirvana. I just don't know who you are!
Part of the problem is the seemingly unending types of documents that are required (e.g., enterprise architecture plans, statements of work, conceptual models, logical models, physical models, implementation models, program documentation, system documentation, user documentation, technical documentation, test plans, training plans, mitigation plans, and so on).
For some unknown reason, we don't go ballistic when we inherit systems with no documentation. We shirk it off as "standard operating practice." Management signs off on deliverables, willing to accept empty promises of "We should have the documentation ready soon." They have to realize that's about as likely as Elvis performing tomorrow night in Las Vegas. Don't be fooled by imitations!
From the IT perspective, if we realize that documentation is critical to our success, then it is the core component to successfully supporting change management. Technically, everything we do is because of change management, regardless of whether its actually managed. Now let's take the business person's view of change management, which is to prioritize timely support for corporate activities and initiatives set about by executives to keep the company competitive and profitable.
We have uncovered the primary cause of what is commonly known as "legacy systems." We are now in a position to create a definition. Legacy systems can be defined as automated processes that lack formal, maintained, accurate, detailed documentation for applications and components of the enterprise, regardless of the language or platform. If this hypothesis is true, it means we are replacing legacy systems with legacy systems, and not legacy systems with client/server. This means that the age of a system is not a defining characteristic of legacy systems.
As I see it, formal detailed documentation (if accurate, maintained and accessible) is the only silver bullet we will need! Should we stop here? Could we further imply that legacy means tools such as CICS, COBOL or ES/9000s? I don't believe so. All of these products are part of the client/server paradigm.
Let's examine why we are spending billions of dollars on the Year 2000 problem, instead of just a million or two. In your best Mr. Rogers voice, can you say "documentation." I knew you could! We're not just doing analysis of a problem, we're on a scavenger hunt; and it's camouflaged, with detrimental consequences if we don't find what we're looking for. Will we get a second chance? We certainly won't deserve one if we don't change our habits.
I suggest a new phrase be introduced for organizations that have appropriate detailed documentation that allows orderly and timely change management. "Power Doc'd." So, if someone tells you their system is "Power Doc'd," you'll know it's a well-documented system and one that might be worth working on or inheriting, should the situation arise. Or, if you are interviewing for a new job, you could proclaim that you "Power Doc'd" x number of systems.
The company I work for has six guiding principles for software development:3
- Define the job in detail.
- Involve the right people.
- Estimate the time and costs.
- Break the job down.
- Set up change procedure.
- Agree on acceptance criteria.
These principles won't work if the documentation is missing or inadequate. It is estimated that between 30 and 70 percent of all new client/server projects are canceled because of complexity, training difficulties, lack of salability, poor systems performance, rising costs, distributed-system surprise, n-tier breakdown, lack of support from corporate management, expectations set too high, etc. The common thread is the lack of adequate detailed documentation. With documentation, we choose the appropriate sequence to develop and migrate systems. We can manage projects. We can educate. We can anticipate. We can perform capacity planning. We can curb expectations. We can situate ourselves for change. Lack of accurate detailed documentation is not something that should easily be brushed off.
The backlog of systems development work is going to increase. If we weren't able to keep up with the backlog in the past, we won't find a panacea to keep up with the pace using visual development tools. What can our users expect of us in the future a larger backlog? The rate of change corporations want in their computer systems is only going to escalate. The organizations that are "Power Doc'd" are going to win this race.
IT must adopt a document-centric paradigm. The ideal starting ground is the Zachman Framework for Enterprise Architecture.4,5 Many of you are probably familiar with its existence, but have not used it in practice or seen it used as a mechanism for change management. For the Framework to be successful, it may require an unorthodox sponsor the business community instead of IT. Because of IT's incentives and motivation factors, IT generally has a narrow focus, which leads to sacrificed quality (documentation) for time and function.
The Zachman Framework accommodates documentation that is needed by all sectors of a corporation that vie for information resources. It is laid out on a 6x6 grid. The horizontal axis contains the interrogatives: what/data, how/process, where/technology, who/organization, when/time and why/motivation. The vertical axis offer perspectives: the business scope, business model, information systems model, technology model, technology definition and implementation. Each cell is unique, and its documentation deliverables don't overlap.
To slightly mis-quote John Zachman, "The end state vision is enterprise-wide, horizontal and vertical integrated documents at excruciating levels of detail." This end-vision is ordered, structured, accurate and living. We can trace the progression and transformation of business rules. We can account for all decisions. We will know why we are doing something. We can prevent reactive on-the-fly solutions. We can even find value-added. I am from England, where there is a saying, "Where there's muck, there's brass." It means that amongst all the junk that nobody wants, a patient person can identify and find riches. Although it appears we just have muck, with a little effort and management, there will be brass in the documentation the riches for change.
The Framework is an important component because it simplifies the management of documents. Even more important than the Framework, and more important than the quality and detail of the documentation, is the accessibility of the documents. Do any of you know of documents that are unavailable because they are on someone's local hard drive, who is on vacation this week, and the workstation is password protected. Maybe your organization has detailed information saved in a CASE tool file format that can't be read by 99 percent of people because they don't have access to the CASE tool. Or paper documents that have for some unexplained reason have gone astray in the library, and you don't know who was the last person to check them out.
"Friend, do not keep your knowledge to yourself; we are a large party; and any benefit which you confer upon us will be amply rewarded." This quote from Plato's Republic, shows that even the ancient Greeks understood the necessity and benefit of documentation! I once heard William (Bill) G. Smith describe the state of our documentation as "tribal knowledge." Over the years, your organization has built up its "tribal knowledge," a term he used to imply that documentation exists, but only in the heads of the employees (or employee, as the case may be). If we examine our aggressive development schedules and couple it with a moderate turnover rate, we can see that IT is facing an information dilemma. Reality check: IT no longer has a single, individual person to use as a central source for its concise, accurate, integral answers. Unfortunately, during the past several years, many answers have been forgotten and have literally walked out of the door, never to be seen again.
Systems development is not rocket science, but its not child's play either. As IT professionals, we have many methodologies and technologies from which to choose. We are continually searching for the magical silver bullet. Fanfare. Ladies and gentlemen, I am proud to announce that the bullet has been found. Please give a warm welcome to "Detailed Documentation" and his side-kick "Change Management." (Rapturous applause, the crowd goes wild). Quiet please, everyone, quiet please. Will everyone start lining up behind the bullet. Yes, that includes you, chief information officer. Project managers, come on now, business specialists, analysts, yes, the designers too. Testers, come along now. Despite our previously bad habits, we've all been invited to try again! IT has an incredible tolerance for sinners. Are you sure I can't persuade you to take that Hippocratic oath?
We aren't going to get our organizations documented overnight. It will take time. We have a lot of catching up to do, and we're already heavily involved in the next round of legacy creation. But, if we start now, documenting slivers of the systems (see Figure 1) that we are working on, we will get there.
A sliver is a subset of documentation for a Zachman Framework cell. A horizontal sliver indicates scope of the documentation, and a vertical sliver indicates the level of detail. Slivers provide a managed approach to growing your documentation. An organization can then set a pace which is manageable and in line with delivery time frames, and then slowly increase the slivers as the enterprise becomes aware of the value in documentation.
In addition to slivers, a data warehouse project is a good way to start building your documentation infrastructure. The warehouse will discover source information of what is important to your business customers. Capitalize on this and keep going. Document the processes; document your new systems.
The day will come when we hear the phrase "migrating legacy to client/server," and we'll be sure that we mean it.
Oh, and as for that rhyme...
For want of a comment, the business rule was lost;
For want of a business rule, the test was lost;
For want of a test, the system was lost;
For want of a system, the customer was lost;
For want of the customer, the business was lost!
1 Gleick, James. Chaos: Making a New Science. Penguin Books. 1987.
2 Drucker, Peter F. Management: Tasks, Responsibilities, Practices. Harper & Row. 1974.
3 Keane, John F., et al. Productivity Management: Keane's Project Management Approach For Systems Development, Second Edition. Prentice-Hall, Inc. 1995.
4 Zachman, John A. "Architecture: Straight From the Shoulder." Presentation. 1996.
5 Zachman, John A. "A Framework For Information Systems Architecture." The ZIFA Letter, Volume One, Number One. 1996.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access