There is a radical movement taking shape in the IT industry. The vanguard of this movement is made up of experienced data warehouse practitioners. These individuals, having seen firsthand the benefits provided by a well-designed data warehouse, have come to believe that the current development process is fundamentally flawed. In fact, they've begun to argue that the current practice is completely backward! To better understand this radical notion, it's necessary to examine the history of decision support technology.
During the 1980s, the relational database came of age. Several of today's largest and most successful corporations were built around relational database SQL (Structured Query Language). Initially, these newcomers focused on transaction processing. They understood very clearly that market acceptance of relational technology depended on its ability to displace mainframe-based transaction-processing systems. If they could successfully open up the OLTP market for their products, financial success would follow.
This strategy paid off. Today, most commercial RDBMS products are capable of processing thousands of transactions per second. However, these OLTP-optimized relational engines and their accompanying normalized designs were not effective at providing answers to real-world business questions. The industry still needed a practical way to deliver the kind of information decision-makers demand.
Data warehousing emerged in response to this need. It proposed the idea of a separate, purpose-built database whose sole mission was to support the decision-making process. As acceptance grew, IT shops all over the world began building their own data warehouses.
These organizations soon learned that the normalized database designs optimized for OLTP are not appropriate for data warehousing. The large number of normalized tables, with their redundant join paths, are too confusing for end users to easily understand. They also result in unacceptably slow query and reporting performance. This insight has led to the widespread adoption of the star schema as the optimal design approach for data warehousing.
The power and flexibility provided by the star schema has led to the heretical notion that the best way to build all corporate information systems, not just data warehouses, is to start by designing stars. This is completely opposite from the way systems are currently developed.
Today, data warehouses are built after their operational counterparts are in place. Designers must often invest huge sums of time and money developing elegant star schemas that will support business reporting requirements and overcome OLTP system shortcomings. However, for all their skill and diligence, even the most experienced designer cannot overcome the all-too-common situation where OLTP systems do not track information critical to business analysts.
The "star-first" movement believes that the current development process is like the tail wagging the dog. They are firmly convinced that decision support is the most important end of the information value chain and that system development efforts should start by focusing on data warehouse design. Only after defining the star schemas required to build a robust data warehouse should development efforts then turn to building OLTP systems.
This is not such a far-fetched idea when you think about it. One of the most compelling reasons for investing in information technology is to provide support for decision-makers.
By designing star schemas first, you are insuring that developers have a complete understanding of the data needed by business analysts before creating normalized designs and building OLTP systems. In this way, the problem of missing data for the warehouse could be entirely avoided.
An additional benefit of this approach comes from the way in which a well-designed star schema closely represents the analytic needs of business people. The clearly defined numeric measures and associated business dimensions are easy to understand and evaluate. As a result, users can provide developers with feedback much easier using star schemas than with entity-relationship models.
Obviously, this proposal for reversing the system development process would require a drastic change to current thinking. However, if reporting and analysis are your highest IT priorities, you might want to consider designing your data warehouse first.
The small but growing group of designers behind this idea is convinced that the "star-first" approach is the way of the future. The question for the rest of us is: Are these people radicals, visionaries or crackpots? Only time will tell. Who knows, the day may come when all development efforts follow the stars.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access