In the past few months' columns we've explored the drivers, frequently asked questions (FAQs) and when to build a federated BI architecture. This month, we'll look in more detail at how you build a federated BI architecture.
- Create a communication forum for the data warehouse (DW) and data mart (DM) teams in your enterprise. A typical large organization may have multiple DW systems and teams, and scores of DM systems and teams, all of various levels of architecture and functionality. The most important thing for you to do in your efforts to federate these stovepipe systems is to create an opportunity, structure and forum for cross-team communication. Your goals are to identify and understand the data content of the systems, the key stakeholders of the systems, and the political drivers and sponsorships of the systems. The business is very sensitive to duplication of effort and to penetration of the various data fiefdoms. You need to pay particular attention to who the political "owners" of the various systems are and to their level of sensitivity to data sharing across the enterprise of "their" data.
- Document your existing DW/DM systems via a high-level enterprise data warehouse architecture (EDWA). The highest level is an entity-level diagram showing the various systems and any existing cross data flow and meta data exchange between them. This investigation and documentation process is accomplished via presentations by each team at your communication forum and by follow-up interviews with each team. During this process, you will discover that many of these systems would not pass any sort of industry-standard test to qualify as a true data warehouse, but the teams are very wedded to this term due to political and budget reasons. You must be sensitive to these issues and not be derisive or negative in any way related to their efforts and their systems. This effort of documenting all the multiple DW/DM systems takes significant investment in time and resources and should not be taken lightly. It is also a prerequisite to achieve the goal of the federated BI architecture. You cannot identify and share critical data across the multiple systems without identifying and documenting it.
- Document each of the existing DW/DM systems at the data flow level. This level includes data flow from each data source, any transformation and integration steps and meta data repositories. In this stage of documentation, you will be identifying each of the data source systems associated with each of the DW/DM systems in your enterprise. Rate each major data element in terms of quality, availability and ease of access. You will also be documenting each of the ETL stages, intermediate data access stages and systems that use the resulting integrated data. It is important to clearly identify the state of quality, aggregation and utilization of the data at this step. You will need to identify data that is in- process data, or data that is being extracted while it is in the midst of an OLTP process. Often in-process data requires special business rules to enable utilization by business users. This is also an excellent opportunity to rate or tag each of the utilization stages as to the political sensitivity of use or sharing with other groups in the organization.
- In conjunction with your users, determine what data offers value-add and impact across multiple systems. This step requires the involvement and commitment of business resources, often a challenging commodity. You must have this commitment of dedicated resources to accomplish your goal, so don't start down the federated BI architecture path without an up-front commitment by your business user community and key top-level stakeholders. The critical characteristic you are looking for is high-impact unique data integration. This data that is formed by the integration of two or more data sources is not available anywhere else in your enterprise and provides politically meaningful impact to the business.
- Collect the various build-phase candidates that derive from Step 4 and analyze them to determine impact and viability. (To help with this analysis there is a free build-phase candidate automated assessment in the resource library at www.egltd.com.) Pick the candidate that provides the best balance between business impact and risk. One of the most important elements to examine is whether the phase candidate advances the strategic agenda(s) of the business and the top-level stakeholders. If your phase candidate is providing nothing more than an interesting data set or something important to a mid-level manager or vice president, it will lack the political "critical mass" to survive the budgeting process and other aspects of corporate politics.
- Implement an enterprise-class ETL tool that supports a common, global meta data repository across the multiple DW/DM systems in your enterprise. An enterprise-class ETL tool is required to provide the backbone of data extraction, transformation and integration required for a sustainable federated BI architecture. The most critical elements gained from the implementation of an ETL tool are a fighting chance to maintain what you have built and some amount of automated meta data population and maintenance.
- Build a small, focused iteration of the federated BI architecture based on the winning candidate from Step 5. Document and publicize success to establish and sustain the political will required for future iterations. Your keys to success in this step are to keep your iterations very small in scope, very focused on meaningful business pain, measurable (you must be able to measure your success) and marketable. It does no good to build a life-saving system for the business if you cannot measure and demonstrate that success through internal marketing and advertising efforts. The goal of the federated BI architecture process, as in all DW initiatives, is to engender and sustain political will in the enterprise. Political will translates into resources and funding for your project(s). In your federated BI architecture iterations, stay small and stay focused on the measurable relief of politically significant business pain.
All Information Management articles are archived after 7 days. REGISTER NOW for unlimited access to all recently archived articles, as well as thousands of searchable stories. Registered Members also gain access to:
- Full access to information-management.com including all searchable archived content
- Exclusive E-Newsletters delivering the latest headlines to your inbox
- Access to White Papers, Web Seminars, and Blog Discussions
- Discounts to upcoming conferences & events
- Uninterrupted access to all sponsored content, and MORE!