Why should this be the case?
First, broadly speaking, the goal of both systems is the same - providing the enterprise with the insight it needs to improve performance. While the EDW may also support various ancillary reporting needs (e.g., the list of customers with aging receivables, or the daily list of orders sitting on some sort of shipping or billing block, etc.), and even serve as a hub for data integration for other applications - a decidedly important function - these really aren't its most compelling purpose. Truth be told, if the warehouse went offline, many of these ancillary purposes typically could and would still be serviced via other avenues. The data warehouse's highest purpose is for the support of managerial decisions that impact a company's bottom line. Period. No system or discipline does this better, in terms of the elucidation of internal, operational factors that are directly under management's control, than activity-based costing.
Second, the ABC system is a data pig. It consumes nearly all the interesting points of data from the data warehouse and/or operational systems just to do its job: customer, supplier and product master data, sales order and billing transactional data, inventory movements and balances, corporate financials and even HR/payroll information. It consumes so much data from so many disparate subject areas that it very nearly, if not entirely, consumes as input everything of real value in the data warehouse itself. It even typically calls for the introduction of many new ancillary data inputs that the enterprise never systematically captured before. So not only do the ABC system and the enterprise data warehouse overlap in terms of their end purpose and goals, they also overlap in terms of data inputs.
These two simple points are bolstered by observations from practical field experience implementing ABC systems. That is to say, the ABC system itself tends to do one of the following:
- Where the enterprise doesn't already have a data warehouse in place, the ABC system becomes the de facto data warehouse.
- Where the enterprise already has a data warehouse, the ABC system often overshadows the data warehouse in business value derived and in sheer data volume.
Third, the data warehouse and the ABC system are very similar topologically. Both tend to be read-intensive querying systems, with long-running batch update/allocation processes that run in the off hours. Both typically involve a fair amount of inbound ETL integration. Both submit nicely to Kimball's dimension bus framework. Both are typically made accessible to end users via the same types of end user reporting tools.
Fourth, keeping the ABC system and data warehouse separate represents a wasteful and redundant expenditure of administrative and infrastructure dollars. We've already discussed that the ABC system will typically consume most if not all of a corporate data warehouse as input. Furthermore, it will generate much more data than that as output. It is simply a waste of valuable processing time and expensive storage capacity to shovel all the data in the data warehouse over to the ABC system, where it will generate even more data, that then gets shoveled back over to the warehouse for publishing. That represents poor stewardship of infrastructure and, more importantly, can delay the business's timely access to the information for which critical decisions simply can't wait.
So the two systems share the same purpose and inputs. They look and feel very similar. And it is unnecessarily costly to keep both of them around simultaneously within the enterprise's application portfolio.
Why keep them separate?
Admittedly, there are some pieces of the ABC system that don't fit neatly within the EDW box as we know it. This mainly comes down to the management tools for the ABC system's metadata. The ABC system involves a significant amount of modeling to set up the rules for how the system should allocate cost from one node in the cost flow network to the next. This modeling, while involving a fair amount of thoughtful business analysis to properly set up, comes down to jockeying around what amounts to a small amount of metadata. The tools which allow the user to create the allocation model, as well as the ABC allocation metadata itself, are probably something that should remain outside the EDW. The other critical components of the ABC system, however, should be merged right in with the EDW.
Pragmatically, what does this merger look like, and what is the implementation path for the enterprise? For the enterprise without an EDW, it is strongly recommended to pursue purchasing or building an ABC system that receives input and generates output at the transaction level. The results realized will be much more valuable and useful, with much faster and greater return on investment, than what would be achieved by constructing a conventional data warehouse.
