Hybrid Approaches to Business Intelligence
Archaeologists tell us that the cradle of civilization consisted of four great river valleys the Nile, the Yellow River, the Tigris-Euphrates and the Indus. We've all heard about the Nile and know of its monumental contribution to Egyptian civilizations, past and present. Likewise, the Yellow River and the Tigris-Euphrates score many historical mentions.
Alas, we have not been able to decipher the language of the Indus valley civilization of 3000 to 1500 BC, leaving it the least documented of all of these important, historical rivers. On http://members.aol.com/Donnclass/Indialife.html#INDUS, Lin and Don Donn provide a unique perspective on the Indus Valley civilization. on their Web site, they feature the following information: We know very little about the civilization that resided on the banks of the Indus, but what we do know is beguiling.
More than 4,000 years ago in the Indus Valley, people built huge cities that had aligned streets and private areas for bathing. Children had toys. All men and women contributed functionally to the society through a system of commerce that resembles a free market system. Their towns were laid out in grids with straight, perpendicular streets and homes that were built to last. These people were incredible builders. They also stowed water in reservoirs for current and future need. Even the smallest house at the edge of each town was linked to that town's central drainage system. It is possible that they not only drained wastewater, but also had a system to pump fresh water into their homes, similar to modern plumbing.
Thanks to modern technology and international rivalry, nearly 1,400 Indus sites have now been discovered. It was a very large civilization large enough to be called a kingdom, but there is no evidence that these people were governed by emperors.
Scientists remain very curious about these people who lived during approximately the same time in as the ancient Mesopotamians and the ancient Egyptians. Did these ancient civilizations know each other? As scientists continue to unravel the riddle of the Indus, we may find that we will have to rewrite history. Was it the ancient Mesopotamians who first invented the sailboat and the wheel, or was it perhaps the people in the Indus Valley?
Today, most successful business intelligence (BI) programs are not built by popularized approaches, but by functional, hybrid, modern approaches.
Some of the first data warehouses utilized a hybrid approach. The people deploying hybrid approaches to BI are incredible builders of data reservoirs ungoverned by emperors, but creating functionality for some very big enterprises with modern plumbing.
The pros of the hybrid approach include rapid development with an enterprise context, avoiding independent data marts, focusing the warehouse on data that provides value not data that happens to be available enforcing consistency of meta data and avoiding redundant extracts and inconsistent transformations through the implementation of a physical data warehouse.
This is really a hybrid of top-down and bottom-up. There are some top-down aspects for items that need to be taken care of up front. However, it aligns with a bottom-up approach whereby there is a repetitive series of brief full life-cycle iterations that continually deliver business value and build on each other.
Hybrid approaches do not make many presumptions or bring a "one-size-fits-all" approach to BI. The up-front top-level enterprise requirements gathering gives the team a perspective as to its ultimate size. This helps to craft an architecture that will meet the longer-term as well as the short-term needs. Revisiting the architecture repeatedly or requiring architecture to still be a main focus years into a program is inefficient.
Data quality has been found to be a huge inhibitor of BI success. A hybrid approach understands this up front and implements a data quality plan. This plan is more than what will "naturally" happen through the coding of the transformations in the extract, transform and load (ETL) process.
The lack of business involvement has been found to be a reason that BI programs are not built to user expectations. Therefore, hybrid approaches utilize the business resource in very tangible ways to compose the data quality and transformation rules, for example, under the umbrella of data stewardship.
With hybrid approaches, the physical architecture layers can be negotiated up front. If the eventual size will dictate a separate physical staging area or a multi- technology data mart layer, they could be implemented early or the program periodically monitored for the inclusion of these layers.
Data sourcing or subject areas make valid iterations in a hybrid approach with the decision based on providing large business value early and often. Through conformed and changing dimensions strategies, hybrid architectures facilitate a high degree of sharing and support a "single version of the truth" for most business subjects by leveraging a physical data warehouse layer that supports enterprise queries but also routes the shared data to the data marts.
Consideration is also given to meta data, performance planning and user training in hybrid approaches. While we may never end up at the Indus river, ultimately successful BI programs do end up as hybrids.