I recently read with interest an article in the Microsoft Architect Journal on so-called Service-Oriented Business Intelligence or, as the articles authors call it, SoBI. The article was well-intentioned but confusing. What it confirmed to me is that plenty of experienced IT professionals are struggling to reconcile Service Oriented Architecture (SOA) concepts with business intelligence.
SOA is certainly a valuable tool in the architecture and development toolbox; however, I think its only fair to keep SOA in perspective. Its an evolutionary technology in IT that has numerous benefits to developer productivity and application connectivity. Im not sure that injecting SOA into a data warehouse environment or framework will do anything more than freshen a few low-level building blocks that have been neglected in some data warehouse environments. Im certainly not challenging the value of SOA; Im just trying to put in perspective to those folks that are focused on data warehouse and business intelligence activities.
The idea around SOA is to create services (or functions, procedures, etc.) that can be used by other systems. The idea is simple: build once, use many times. This ensures that important (and possibly complicated) application processes can be used by numerous disparate applications. Its like an application processing supply chain: let the most efficient resource build a service and provide to everyone else for use. SOA provides a framework for allowing multiple applications access to common, well-defined services. These services can contain code and/or data.
The question for most data warehouse environments isnt whether SOA can improve (or benefit) the data warehouse; its understanding how SOA can benefit a data warehouse.
Weve got lots of clients leveraging SOA to support their data warehouse. Theyve learned they can leverage SOA techniques and coding to deliver standardized data cleansing and data validation to a range of business applications. They have also upgraded the operational system data extraction code to leverage SOA which allowed other application systems (or data marts) to reuse their code.
However, their use of the SOA hasnt been focused on enhancing the data warehouse environment as much as has been focused on packaging their development efforts for others to use. Most data warehouse developers invest heavily in navigating ITs labyrinth of operational systems and application data in order to identify, cleanse, and load data into their warehouses. What theyve learned is that for every new ETL script, there are probably 20 other systems that have to custom developed their own data retrieval code and never documented it. The value that many data warehouse developers find with SOA isnt that they are improving their data warehouse; theyre just addressing the limitations of the application systems.
Evan Levy's blog can also be found at http://baseline-consulting.typepad.com/evanlevy/.