Most companies (midsized and up) have what they would call a data warehouse. However, one of the major arenas of difficulty is in making these data warehouses real time. Operational systems need to cooperate with this vision by not being so fragile that they break with intraday extracts. The data warehouse environment needs to be efficient to the point where the requested extracts are kept to a minimum. These do not always describe the reality, so real time remains a challenge.
Also, BI tools are not as easy to use, or as interesting, as the data warehouse community tends to think. Builders of data warehouses typically do not programmize the rollout enough and leave some utility on the table. And finally, reporting is not always the final answer. If batch-loaded data warehouses built for reporting are the end-game of BI today, it's time to consider a different profile for tomorrow. Event-driven and process-oriented BI is needed.
Information cannot be out of date. Out-of-stock conditions, customer complaints and fraud are not most optimally solved with reporting. Out of stock, for example, needs a reorder placed into the system at the most optimal point, potentially that day or hour. For a large company, fortunes are gained and lost by suboptimal business decision timing. Data validations are another major area where the timing needs to be immediate. Clearly, analytics need to be embedded in the processes. Reorders typically need to incorporate many factors beyond simple threshold analysis. More complex models will look at demand, vendor offerings and storage capabilities to make a decision. This, I suggest, is BI as we know it today.
The former holy grail of information architectures was solely focused on a singular business result that needed to be accomplished. Then, we got quite a bit more efficient by introducing a data warehouse into the center of things - the center of the information universe, if you will. However, some data warehouses remain fixers of tactical information problems, such as the lack of query concurrency, the ability to query the data in the operational system and the need for a historical data store. The trade-off - data latency. And what about data quality and data integration? They sometimes remain a challenge. If you have overcome these challenges as well as the real-time challenge, congratulations - you have bought yourself some time in the pressing matter of information management rearchitecture.
Other operational possibilities include master data calculations. The operational environment is the optimal place to leverage bringing master records together. Operational BI has come to mean certain levels of reporting/dashboarding, etc. in the operational world where integrated data and analytics are needed. I'd like to extend that definition to include event-driven business actions. Also, keep in mind, modern enterprise resource planning (ERP) is peeling analytics back where the demand is - operationally. ERPs are undergoing explosive growth.
The post-operational world is losing function to the operational world as time goes on. Data warehouses have provided a stop gap and will certainly retain some key functions, which I will discuss in the next column. But our thought processes that shoehorn all issues into reporting and all reporting into the data warehouse need to be updated.
Enterprise information integration (EII) is a support mechanism for operational BI. Data can be made to appear in the same data store as the data warehouse if desired - one-stop shopping for corporate information. EII is useful when connecting structured to unstructured data and when immediate data change in response to the data view is desired (i.e., when changing a copy of the data will not suffice). EII has utility when the data transformation is relatively light or nonexistent, and just getting the data together for integrated query is the biggest challenge.
EII query performance needs to be considered, and the relatively worse performance (versus the obvious advantages of physical cohabitation) must be acceptable. However, query performance has for too long been considered a knockout issue, while manageability and maintainability, which I would argue are more important overall than top-percentile performance, never seem to gain such status. I suggest they should.
In the next column, I will continue to argue for selective inclusion of operational BI into our information architectures and some new thought processes around data warehousing.
William McKnight brings process, organizational and architectural focus to building strategies and implementing master data management and data warehousing programs that have consistently, for many years, improved the productivity and performance for his clients including several global corporate giants. McKnight, award-winning consultant and author, can be reached at McKnight Consulting Group.
Be the first to comment on this post using the section below.