Continue in 2 seconds

The Integration of PBI with Existing Data Warehouses

  • March 01 2002, 1:00am EST

Prior to bringing a baby home from the hospital, your other children get all your attention. When the new baby comes home, suddenly the parents' attention must shift to the new arrival. Older siblings learn to adjust, but it takes time. Sometimes, the adjustment is very difficult. Hopefully, the older siblings will eventually accept the fact that they must share their parents' attention, get along with fewer toys and contribute to the raising of the infant.

The new baby, in this case, is a packaged data warehouse. Like a baby, it grows over time. In fact, it may grow to be bigger and more important to the business than the data warehouse you have in place today. However, unlike the time required for a baby to catch up to a four-year-old sibling, it won't take fifteen to twenty years for them to be the same size; the packaged data warehouse can catch up to the bespoke data warehouse in less than a year.

Because most companies claim to have data warehouses and many are purchasing full packaged business intelligence (PBI) or components (defined in last month's column), this month I'll give you some parameters for finding a workable coexistence between the two.

The first area of consideration is a Category 1 PBI such as prebuilt ETL, prebuilt data models or end-user applications. These items alone do not complete a data warehouse, but ideally provide more than eighty percent of the component that they are targeting. The warehouse is filled with other equally robust components. As for the end-user application in particular, the best thing that your data warehouse can do is to provide the data that its underlying database needs from the data warehouse/mart or map its data needs to the data already in, or added to, the data warehouse/mart. That's where the clean, historical data for an organization should be found.

It can't be emphasized enough that many IT organizations need to accept the new "buy" world and accommodate its needs by focusing their attention on these issues of integration and architecture. For example, the perceived aloofness of many data warehouse organizations is causing businesses that want the applications to source the data from the operational systems themselves, causing inefficient, redundant extracts ­ probably with inconsistent transformations ­ to permeate the organization.

The second area of consideration, one which requires more attention, is the integration of Category 2 PBI (sourcing, data model and access layer) such as SAP BW or PeopleSoft EPM. This is far trickier. Ideally, over time, with their price points and need for attention, these offerings will become what you consider your enterprise data warehouse. This will not happen overnight, and there should be a process to get from point A to point B that does not deny the value of the existing warehouse. Applications, feeds, subject areas, etc., should be moved to the PBI over time. However, there may be functionality that the PBI will never provide, thus creating a dual warehouse environment.

Organizations that have integrated Category 2 PBI offerings successfully with an existing enterprise data warehouse environment have done so not by starving the legacy warehouse of attention, but by incorporating elements of the warehouse that are working and components that are valuable to the long-run needs of warehousing in the organization. These are elements such as feeds, transformation rules, data marts and modeling constructs.

By any name, packaged data warehouses, packaged business intelligence, analytical applications, data warehouse jump-starts, packaged applications or packaged data warehouse infrastructures never completely meet the data needs of an organization out of the box and must always be customized. Inevitably, some of the required customization can be found in the legacy data warehouse of an organization, which may perhaps be right under your nose.

My next column will discuss the things to look for when purchasing PBI components or a complete solution and how to make your purchases exceed your expectations.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access