To anyone who has been awake and involved with information processing for the past decade or two, it comes as no astonishment that there are many types of databases. In the beginning the database theoreticians called for a single database to serve all purposes. This was no surprise as the theoreticians were at that time focusing on cleaning up a world where there was nothing but applications built for very immediate requirements. But when you look around today, you see all sorts of databases which do not fit the original theory of a single database to suit all purposes. We have transaction processing databases. We have super high-performance databases. We have data warehouse databases. We have exploration databases. We have archival databases. We have data mining databases, and so forth.
How exactly do these databases fit together? What framework brings these very different types of databases and their associated disciplines together to form a cohesive whole?
The Chain of Beneficence
In order to understand how the different kinds of databases fit together in a coherent manner, consider a structure that can be called the "chain of beneficence." This chain is shown in Figure 1.
The chain of beneficence describes how ideas are turned into applications. The chain connects several related activities. The major activities are:
The chain of beneficence appears to be vaguely similar to the old notion of a waterfall development methodology. While there indeed may be some similarities, there are some very distinct differences as well. Some of the important differences between the chain of beneficence and the waterfall methodology are:
1) There is passage from any one link in the chain to any other. For example, an analyst may be observing some data. The observation may trigger an inspiration. The inspiration may lead to more observations, at which point the analyst may create a hypothesis. The hypothesis is examined and rejected. The analyst returns to observation and re-hypothesizes the observations. Now the hypothesis is examined and predictions are made. In such a manner the chain of beneficence is traversed.
2) The chain of beneficence can be entered or exited at any activity. In other words, any project that passes through the chain of beneficence can start anywhere and can end anywhere. This describes the reality of projects that sound like a good idea but simply do not pan out. Or the same phenomenon describes projects that are created with no theoretical foundation.
The maximum benefit accrues when a corporation applies the chain of beneficence from inspiration to exploitation.
Of interest is the fact that the chain of beneficence is served by different kinds of databases. The different types of databases which serve the chain of beneficence are the exploration warehouse, the data mining warehouse, the classical application database and the data warehouse. Figure 1 shows how these databases serve the different activities.
But there is another perspective of the databases. That perspective is shown by Figure 2.
The Circle of Transformation
Figure 2 shows that there is a circle of transformation among the different databases. The application database provides data for the data warehouse. The data warehouse, in turn, provides data for both the data mining warehouse and the exploration warehouse. In between the different types of databases is a transformation process. Each of the transformation processes is fundamentally different. But each transformation process indeed passes data from one environment to another and makes a fundamental transformation of the data. Note that the databases form a circle in which data is repeatedly transformed and recycled. Of course, the circle of transformation operates in sync with the chain of beneficence.
The chain of beneficence is not a closed-loop system. Projects enter and exit at any point. In addition, projects can go from one link in the chain to any other link in the chain. There is no standard flow within the chain of beneficence.
In addition, the chain of beneficence is not a methodology. A methodology has prescribed steps, prescribed flow and well-defined activities. Imagine the difficulty in trying to prescribe that someone have an inspiration. Or even more difficult, how to regularly have an inspiration. It is doubtful if anyone on earth at anytime has called for or expected a regular occurrence of inspirations. Instead, the chain of beneficence describes the relationship between processes and how those processes can meaningfully relate. And, in doing so, the chain of beneficence provides insight into how the underlying data relates.
The chain of beneficence provides a 50,000 foot perspective a truly high- level perspective of how different components of the corporation relate.
One of the interesting issues is how formal the chain of beneficence and the circle of transformation should be. In some places, the chain is very formal. Between the application database and the data warehouse there is a set of extremely formal transformation procedures. Between observation and inspiration there is the flimsiest of formal interfaces. So the chain of beneficence runs the gamut when it comes to formalization of interfaces. In some places, the interfaces are very formal and tangible; in other places, the interfaces are practically non-existent.
The exploration warehouse plays an interesting role in the chain of beneficence. The exploration warehouse becomes the spawning ground for a wide variety of activities. Inspiration can occur in the processing that takes place in the exploration warehouse. Certainly observation occurs in the exploration warehouse. And even testing and formalization of the hypothesis occur in the exploration warehouse. The exploration warehouse becomes the incubator for many early activities in the chain of beneficence.
It is true that applications can be created without passing through the activities outlined in the chain of beneficence. Applications have been created over the years with no knowledge whatsoever of the chain. However, the odds of success and the chance that an application is going to have an important business payoff are mightily increased by passing through one or more iterations of the chain of beneficence.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access