for Information Management Blogs
APR 8, 2009 2:52am ET

Blogroll

Blurring the Line Between SOA and BI

Print
Reprints
Email
I recently read with interest an article in the Microsoft Architect Journal on so-called Service-Oriented Business Intelligence or, as the article’s authors call it, “SoBI.” The article was well-intentioned but confusing. What it confirmed to me is that plenty of experienced IT professionals are struggling to reconcile Service Oriented Architecture (SOA) concepts with business intelligence.

SOA is certainly a valuable tool in the architecture and development toolbox; however, I think it’s only fair to keep SOA in perspective.  It’s an evolutionary  technology  in IT that has numerous benefits to developer productivity and application connectivity.  I’m not sure that injecting SOA into a data warehouse environment or framework will do anything more than freshen a few low-level building blocks that have been neglected in some data warehouse environments.  I’m certainly not challenging the value of SOA; I’m just trying to put in perspective to those folks that are focused on data warehouse and business intelligence activities. 

The idea around SOA is to create services (or functions, procedures, etc.) that can be used by other systems.  The idea is simple: build once, use many times.  This ensures that important (and possibly complicated) application processes can be used by numerous disparate applications. It’s like an application processing supply chain:  let the most efficient resource build a service and provide to everyone else for use.   SOA provides a framework for allowing multiple applications access to common, well-defined services.  These services can contain code and/or data.   

The question for most data warehouse environment’s isn’t whether SOA can improve (or benefit) the data warehouse; it’s understanding how SOA can benefit a data warehouse.  

We’ve got lots of clients leveraging SOA to support their data warehouse.  They’ve learned they can leverage SOA techniques and coding to deliver standardized data cleansing and data validation to a range of business applications.  They have also upgraded the operational system data extraction code to leverage SOA which allowed other application systems (or data marts) to reuse their code.

However, their use of the SOA hasn’t been focused on enhancing the data warehouse environment as much as has been focused on packaging their development efforts for others to use.  Most data warehouse developers invest heavily in navigating IT’s labyrinth of operational systems and application data in order to identify, cleanse, and load data into their warehouses.  What they’ve learned is that for every new ETL script, there are probably 20 other systems that have to custom developed their own data retrieval code and never documented it.  The value that many data warehouse developers find with SOA isn’t that they are improving their data warehouse;  they’re just addressing the limitations of the application systems. 

 Evan Levy's blog can also be found at http://baseline-consulting.typepad.com/evanlevy/.

Filed under:

Advertisement

Comments (4)
Great article! My sense is that instead of having this kind of heavy-lifting with million $ softwares like Cognos, BO, etc ... many companies would use the downmarket environment and create home-grown ETL tools [customised and easily customisable] probably using the same technology and existing people that they are using for other Enterprise tools and ask the IT teams / outsourced company to do it at a lower cost / fixed price contract.

After all, cost is key here as many cannot afford INR 4 million [in India] tools from Cognos, et al.

So I expect to see lots of stuff specially using .NET [SQL server already has so much in it SSIS, SSAS, SSRS] and JAVA / Oracle and other technologies.

PHP probably will be used for a lot of web analytics BI.

Just my 2 cents, Regards, Arup Bhanja learning Analytics and BI

Posted by Arup B | Friday, April 10 2009 at 7:24AM ET
I agree, an excellent article. I don't see that SOA has an enormous amount to offer BI, from a delivery perspective at least. The greatest benefits are going to be down in the data integration layer. As you've pointed out, the process around data cleansing etc are prime candidates, as are processes for tieing data sets into metadata layers and accessing master data. But then, the problem for many people is that they get blinded by the bright lights of BI delivery without considering the effort required to get the data there in the first place.

Cheers!

Posted by Rod S | Tuesday, April 21 2009 at 8:11PM ET
Add Your Comments:
You must be registered to post a comment.
Not Registered?
You must be registered to post a comment. Click here to register.
Already registered? Log in here
Please note you must now log in with your email address and password.

Blog Archive for Evan Levy

The Time Has Come for Enterprise Search
The Problem with Total Cost of Ownership
Complex Event Processing: Challenging Real-Time ETL
The Flaw of the Data Inventory
So You Think You’re Ready for a Data Warehouse Appliance, Part 2

More from Evan Levy »

Blog Index »

Twitter
Facebook
LinkedIn
Login  |  My Account  |  White Papers  |  Web Seminars  |  Events |  Newsletters |  eBooks
FOLLOW US
Please note you must now log in with your email address and password.