Over the last several years, the data warehousing and business intelligence (BI) industry has seen multiple dynamics emerge and subside. From a technology standpoint, it saw a growing number of tool-based vendors emerge, and last year it witnessed significant consolidation. Similarly in the approach toward executing BI projects, the industry began with companies going “big bang” in terms of building their enterprise data warehouse all at once, only to realize later that data warehouse projects are best executed by breaking them into multiple small and manageable phases.
In today’s BI market, one of the buzzwords making the rounds is “frameworks.” The Wikipedia definition of framework is “a basic conceptual structure used to solve a complex problem.”1 As part of this industry, I suppose all of us would agree that BI is a complex problem to solve. So, going by the above definition, the usage of frameworks in this space is absolutely relevant.
A BI framework is a group of reusable components that can be leveraged while developing a BI subsystem. I am consciously using the word subsystem because a framework can be applied to the extract, transform and load (ETL) layers, the underlying data model or even the front-end analytical layer. The most important attribute to understand is that a BI framework can be a concept and does not have to be something tangible or physical.
I will give you an example of a framework that is a concept that does not exist in physical sense but can be leveraged in a BI subsystem design.
In the U.S., the Securities Exchange Commission mandates all asset management companies to file what they call 13F/G/D reports. These are regulatory reports that need to be submitted every quarter and deal with a declaration of the quantity of holdings individual advisors have in specific security classes. (Note: The penalty for noncompliance can usually run into the millions of dollars). From an IT standpoint, rendering these reports involves humongous effort toward collating the data from a wide variety of sources, cleansing it, correcting it and arriving at the correct number. An obvious question is, “Are there any existing frameworks that can help me accomplish this for my business?” If the answer to the previous question is “Yes,” what does that framework look like?
In this particular example, that framework is a concept. The concept is made up of detailed understanding of how an asset management company carries out its business (domain) and a generic data model that supports rendering 13F/G/D reports, which will require customization to make it work in your organization.
To understand the concept you must ask:
- What does 13F/G/D mean from an SEC standpoint?
- What are the data elements involved?
- What are the sources of these data elements from a systems standpoint?
- What are the known issues you will face when it comes to collating all these data elements into one single report, and how do you solve them?
Imagine yourself trying to execute the project with the answers to these questions versus starting from scratch. Would you have a higher chance of successful execution? The point I am driving at is that in the BI domain, knowledge itself can be viewed as a framework and leveraged.
What Frameworks Can Achieve
The primary driver toward using frameworks is obviously to reduce cost. The unique selling position for using frameworks is that it gives the user a prebuilt platform to start from, and the savings in cost comes from the effort saved using a prebuilt platform.
