An obvious mistake architects and project sponsors make is associating BI and warehouse efforts to particular technologies. Too often, the notion of implementing an online analytical processing (OLAP) tool or building a star schema overshadows the rationale and purpose for these tools and techniques in the first place. Hopefully, it was a business requirement that drove project planners to recommend the use of OLAP, for example. It is important to understand that business requirements drive warehouse iterations and not the techniques used or technology implemented.
Nevertheless, the data architect or solution strategist must recognize when a particular technique or technology will be necessary. Moreover, these individuals must plan the overall BI architecture in advance of any iteration being implemented. This includes any data structures required to support the overall BI plan as well as predicting the type of technologies to implement. Architects and solution strategists must set into motion an inclusive picture, an encompassing agenda for virtually any combination of BI applications that might be necessary for their particular enterprise.
The core reason for establishing your BI vision in the form of a strategy document is to ensure that implementation of specific technology or a data structure (e.g., data mining or the establishment of an ODS) is not done in a haphazard manner. The project planners must understand what and where the technology or technique fits in their overall BI environment.
Implementing your BI organization as a big-bang effort has long been deemed a formula for disaster. Instead, BI environments are grown iteratively, addressing business requirements one at a time. Iterations, consequently, must remain focused and well defined to address specific business requirements. On the other hand, iterations must be assimilated into the long-term vision of the BI organization. Iterations are driven by specific requirements but guided by the broader, enterprise-wide road map. The conceptual diagram of the business intelligence organization as shown in Figure 1 serves as the basis for such a road map and all subsequent planning, including the basis of the strategic document.
Figure 1: The BI Organization Conceptual Architecture
Components of Your Strategic Document
There is often confusion regarding the content required in a BI strategy document. Much of the perplexity is due to the several competing methodologies for building data warehouses and supporting BI initiatives. From industry leaders to vendors, they all offer some approach that has proved successful. However, in all the various approaches and methodologies, there are a few topics that everyone agrees must be in your plan, including some variation of the following four views or components:
- Conceptual View
- Data Architecture
- Technical Architecture
- Implementation View
Each topic contains numerous opportunities for detail to be introduced. Even a high-level perspective such as the BI organization diagram (Figure 1) provides an excellent road map of your enterprise initiative and an opportunity for architects and project planners to define and describe, in some detail, the individual components.
There are some aspects of the overall strategy that can be addressed in detail, while other elements can only be dealt with at a very high level or conceptual perspective. However, because your BI strategy document is a "living" document, you will be able to continuously refine your enterprise road map with subsequent iterations of your warehouse. It should be your intent to update, modify or otherwise evolve your enterprise plan as you implement project iterations and more detail becomes available. For example, let's assume that we know data mining will be needed at some time in the future. We need to emphasize that fact and establish the standard for its use. However, we probably do not have any other detail about the mining tool or technology necessary. Consequently, we will wait until we have a specific iteration that would trigger such a decision to document the exact software to be implemented.
As previously mentioned, there are at least four major sections to your BI plan: conceptual view, data architecture, technical architecture and implementation view. Within each section, there are numerous topics that you could cover, from conceptual diagrams, to use-case views, to security to standards. What you ultimately include under each section will be largely driven by the size and scope of your BI initiative, your ability and authority to establish guidelines and your experience. The following content will describe some components you should consider for each core section when drafting your overall BI strategy.
There are at least five issues that can be addressed when defining your enterprise strategy within the conceptual view section, including:
Architecture goals and constraints. This topic provides a means for architects and planners to set overall goals and objectives related to the BI initiative. The establishment of any constraints that impact the BI effort must be defined as well. For example, the BI initiative may not be able to start until the new data center is complete, or the overall BI effort needs to begin with finance or our offices in the Pacific Rim.
Conceptual view. The BI organization conceptual diagram in Figure 1 is the type of view expected in this section as well as a brief discussion of each component illustrated.
Use-case view. A use case can be developed in order to formalize the intention of the BI initiative.
Architecturally significant design components. There may be any number of significant components that are part of your overall architecture design. Perhaps you are planning for extensive spatial data and analysis -- that is the level design significance that should be mentioned here. Or, your enterprise may need an ODS for tactical reporting, or you may be planning to implement a message broker or XML server as a data delivery mechanism for the warehouse. These are all examples of significantly important design components.
Standards. This applies to any standards that warehouse participants must adhere to. For instance, a standard may be to use a BI community to give final approval to any particular iteration sought.
The conceptual view provides a broad overview of your entire BI vision; it represents the "what" part of the strategy document.
The data architecture, as shown in Figure 2, provides designers a venue to convey what data structures are going to be implemented, how that data is stored in each and how the data will propagate throughout the warehouse environment. Four possible subsections to support your documentation of the data architecture follow.
Figure 2: Data Architecture View
Data architecture goals and constraints. All the goals, objectives and constraints to your strategy should be documented in this section. For example, the goal might be data integrity. Planners might identify the following objective in pursuit of the goal: All warehouse-centric data must be incorporated into the atomic layer first. All subsequent use of that data will be sourced from this single data structure. Constraints might include third-party data or technologies.
Logical data architecture. This is your opportunity to provide logical models that support your data architecture goals. Remember that initially, you will be limited in the models that you can provide because you are not yet addressing a specific warehouse iteration. Therefore, you could provide a subject area model of your enterprise or a series of subject area models that describe core subjects within your enterprise. You can include rules for mutating raw source data into atomic-level data and even guidelines defining how and when to use star schemas and OLAP cubes.
Architecturally significant design components. The establishment of an atomic layer is a significant architectural component, an ODS is another as is an enterprise cube farm. These are traditional design components. However, there are others, such as: geospatial data structures, specialized data staging and "living" warehouse databases.
Test plans. A component of the overall road map that is often overlooked is how iterations will be tested before rollout. This section provides planners an opportunity to establish a standard to follow for all subsequent warehouse iterations with regard to testing and acceptance. Topics should include: test templates to be completed by future project sponsors and planners, criteria for enterprise adherence and approval, criteria for test data selection and performance testing (including unit, suite and stress testing).
Data architecture view is obviously a critical section for any warehouse-centric initiative. As with the technical architecture, you will often start at a high level and grow the details of your architecture as successive iterations of the warehouse are undertaken.
When you first start to lay out your technical architecture, it may be limited. For instance, you may know that you want to implement star schemas in a relational database. What you may not know by the time the first draft of your BI plan is published is whose relational database management system (RMDBS) you are going to use or on what platform it will be implemented. That's okay. What is important is to start designing your architecture, including detail where possible, and ensuring placeholders where subsequent decisions will provide more information. That means you will revisit the technical architecture many times and publish updated versions as necessary to ensure communication to your audience. As shown in Figure 3, the technical architecture diagram gives a reader a general sense of the architecture as it is currently known and yet ensures a place for additional information as it becomes available.
As with the overall strategy, the technical architecture section can contain many topics. Following are a few to consider.
Technical architecture goals and constraints. The technical architecture focuses on tangible components of your BI environment. The goals, objectives and constraints outlined in this section should be specific to those topics.
Technical architecture. This diagram specifically identifies the hardware, software and network/communication components. Figure 3 illustrates how technical components of a BI environment are represented in diagram form.
Figure 3: High-Level Technical Architecture
Architecturally significant design component. Any significant technical component of your architecture must be identified in this section. For example, you may require a 7x24 implementation and therefore will establish mirroring across two distinct data centers. That would be considered significant.
Components of the technical architecture must be described sufficiently. This often means that your technical architecture documentation includes several technical diagrams and many pages of related narrative.
Within this section, designers and project planners begin to establish the guidelines necessary for building and maintaining the purposed warehouse structures and related technologies. The implementation of core processes and the sequence of establishing data structures are detailed in this section. As with the previous sections, the implementation view can start from a high-level perspective, with detail added to this document as it becomes known. Generally, the implementation view will contain three distinct perspectives: the overall strategy, architecture and process.
Strategy: The implementation view might cover a time span or discuss when resources will be available for BI-centric projects. Here, you can formalize how you plan to identify and prioritize BI iterations using process flows such as that shown in Figure 4. Finally, this section should outline how funding will be addressed.
Figure 4: Implementation View
Architecture: This perspective includes topics such as size and performance requirements, data quality issues, meta data control and retention policies. Decisions made on retention, for instance, will impact data architecture issues such as partitioning of the data as well as technical architecture considerations regarding disk storage.
Process View: Here, the architect must outline, at a high-level, process issues such as: refresh rates, backup/recovery, archive, workflow and security. Again, you will address as many of these topics as possible even though no particular iteration of the warehouse is being discussed.
Executing Your BI Vision
Your BI plan should start with high-level diagrams, broad policy statements and general definitions. As your BI environment matures, so too will the formal documentation and depth of detail identified in your strategy document.
The BI organization is a planned, architectural vision for your enterprise. The BI strategy document is the necessary road map to follow as you begin implementing your environment, iteratively. The road map must be tuned and constantly adjusted to reflect the needs of your business and its strategic direction. Without a broad architectural vision, BI iterations are little more than haphazard warehouse-centric implementations that do little to create an enterprise-wide, informational asset.
Your strategy document offers insight into the BI environment. It must focus on communicating what you are planning to build, how you plan to build it and when users can expect their requirements to be met. To that end, a BI strategic document will include at the very least four core components: conceptual view, data architecture, technical architecture and implementation view. Each of these components provides readers with a unique perspective of the BI environment being planned.
Acknowledgement: Content for this article is expanded in the book IBM Data Warehousing by Michael L. Gonzales (published by Wiley).
Michael L. Gonzales has been a chief architecture and solutions strategist for more than a decade, specializing in business intelligence technologies and techniques. Gonzales is currently a Principle at Claraview, Inc., where he leads the Education department teaching a series of DW/BI courses internationally. He is a successful author; his latest book is BI Strategy: How to Create and Document. You can reach him at email@example.com.