It wasn't supposed to be this way. The promise of the enterprise resource planning (ERP) systems and the emergence of enterprise application integration (EAI), data warehousing (DW) and business intelligence (BI) technologies was supposed to revolutionize the way business was conducted, or at least the way systems supported streamlined business processes. Why, to a large degree, has the promise of these technologies gone unfulfilled? Why do so many organizations have so much trouble accessing, coalescing and understanding their corporate data in a timely manner? In short, why is data management and analysis still such an intractable problem?
It's not the fault of the technologies; it's the way they're being used. The concepts of data integration and BI are still viable, but not in the way you might think. It's not applications that need to be integrated or centralized, per se. It's the information contained within those applications - the information used to support business processes, analyze performance and predict the future - that needs to be integrated and made available, as it is needed, to knowledge-workers and decision-makers.
True enterprise-wide information integration requires an information strategy that combines the concepts of business process management, time-driven business intelligence and information integration concepts into an architecture that spans the enterprise in business time. Let's call that solution the business-time information architecture (BTIA).
Why business-time instead of real-time? Isn't real-time data availability the golden chalice of IT these days? It is and it isn't. Businesspeople often need information in real time or near-real time. Other times, semi-latent information or even historical information will do. The concept of business time recognizes that users' needs aren't always covered by access to the most current data. Figure 1 represents the data volume and need-time intervals for common business events.
According to Holly Hayes, program director for Information Integration at IBM's Silicon Valley Lab in San Jose, California, businesses need contextual comparison ability as well as access to real-time data. "You need data from different sources and in different contexts," says Hayes. "Take business activity monitoring, for example. Business rules are triggered when certain events occur. Organizations need to be able to look at both a specific event as it happens as well as look at it in the context of what has happened historically."
Hayes maintains that with a properly integrated architecture, users have access to the data they need, regardless of time context. "There is access to both historical and real-time data," she says, "But the information is presented to the user in a similar format where he or she can compare the historical data with current data."
Figure 1: Need-Time Intervals for Common Business Events
The information architecture that enables this common view of diverse data is the BTIA. The BTIA employs technologies such as enterprise information integration (EII) tools and BI engines to enable organizations to integrate content from multiple sources such as data warehouses, XML documents, ERP systems, legacy systems, operational data stores and content repositories. That data is combined through a transparent semantic layer for analysis and decision making. The information is then presented to the users in need-based views through an enterprise portal or a customized BI interface.
With its focus on information integration, the BTIA facilitates business process management, enterprise performance management, enterprise analytics and business activity monitoring. These activities are critical to the continued competitiveness and success of any organization.
So much for the theoretical definitions. Let's look at the critical components of the BTIA and how they function together to facilitate enhanced information integration and analysis.
The first, foundational, component of the BTIA is a best-of-breed EII solution. EII is a fairly recent concept; many tool suites are still in their first or second releases. However, due to the emergence of some robust EII packages, EII is beginning to come into its own as a mature technology.
EII tools differ from vendor to vendor in how they integrate organizational information. Some vendors, such as New York, NY-based MetaMatrix, Inc., use meta data modeling tools and a meta data repository to create a virtual database that sits as a transparent layer over, and virtually connects, the organization's disparate systems.
Other vendors, IBM, for example, with its DB2 Information Integrator 8.1, vary this concept. They use a federated data server plus a replication server. Source-to-target mappings are created and stored in a federated data catalog to give users the ability to create "views" that mimic SQL tables. However, the result is the same for either method. The technology creates a user-transparent semantic layer that enables users to have access to unprecedented amounts of organizational information.
EII does not take the place of the organization's operational data stores (ODSs), data warehouses or data marts. There is still a need to rationalize, normalize and integrate historical data from many sources. What's great about EII is that it leverages the entire information universe of the organization by enabling businesses to extract, on an as-needed basis, information from these and many other systems for reporting and analysis.
The next essential component of the BTIA is a best-in-class BI engine to make sense of all the data that's made available through the EII solution. Unlike EII, BI is not a new concept. BI technologies have been around for several decades and have enabled many organizations to improve their information analysis capability; however, BI technology can really reach its full potential when combined with EII technologies as part of the business-time information architecture. Figure 3 illustrates where EII technologies fit within the BTIA.
Figure 2: Theoretical Design of the BTIA
The EII and BI technologies weave together all the other components of the BTIA including business activity monitoring (BAM), business process management and performance management/monitoring. They are the core of the architecture. With the core pieces in place, let's take a look at the functionality the BTIA provides.
The first function of the BTIA is to enable users to access and analyze data from sources other than traditional SQL databases. It also enables them to perform operations on the resulting data sets as if they were in a database. EII technology is absolutely critical to this functionality.
Indeed, IBM's Holly Hayes believes that EII is the key enabler of multisource data access. "I may be accessing an XML document, data from a data warehouse and data from a legacy system," says Hayes. "I map that data from those different sources into what appears to the end user to be a relational database schema. Everything that you see looks to the application layer like a standard SQL view. You can apply the standard SQL expressions - filtering, derivation, calculations, selects, joins, etc. - to all of those back-end objects using standard SQL."
The BTIA then uses a transparent semantic layer that allows information from multiple sources to be presented to users in a common format. The integration process is completely imperceptible to users and is delivered through a presentation layer such as an enterprise portal. Says IBM's Hayes, "EII technology is transparent to the user. What's great about EII, then, is that you can use it within the context of standard BI tools, such as those from Business Objects and Cognos."
By leveraging EII technology, the BTIA also enables organizations to make sense of their meta data anarchy and harness the power of meta data to grow the business. According to Rob Cardwell, chief technology officer at MetaMatrix, better meta data management is the key to bringing order to meta data chaos and integrating information to drive business success. "With proper management, meta data really becomes part of your execution environment instead of just an after-the-fact populating of the repository with captured meta data," says Cardwell. "The value proposition is that if you manage meta data correctly, you can use that meta data to drive your run-time activities and turn your meta data into an executable asset."
Figure 3: EII's Place in the BTIA
Solving the meta data chaos with EII enables the organization to speak with one business vocabulary and to obtain a single version of the truth about organizational data. As Cardwell puts it, "EII allows organizations to achieve a common vocabulary by providing a virtual layer on top of all those different physical systems. What this means is that whether it is customer number, order number, or facility ID, the data is described the same way throughout the organization."
Darren Cunningham, a product marketing director at San Jose, California-based Business Objects, agrees. "Different users need different versions of the same set of information, but with the same version of the truth," he says. "Coupling integrated information with BI makes that critical data available so that different people with different needs can access it."
How important is speaking with one voice? Business Objects' Cunningham says a common vocabulary is a catalyst for business improvement and growth. "When the entire organization has one single version of the truth about organizational data, there's going to be business improvement by developing and adhering to a common business vocabulary," says Cunningham. "That [vocabulary] might include everything from on-time delivery to net margin, but now, across the organization, these fundamental measures can be measured in the same way."
The BTIA also incorporates business process management into the information infrastructure. By configuring the BI component of the BTIA to fit their management philosophy, organizations can use the architecture to drive their processes. Cunningham agrees. "A good EII/BI combination can be configured - with statistical process control maps, for example, that are put in a dashboard - to conform to the metric definitions of a particular methodology," he says. "This flexibility enhances the process and quality management strategy of the business."
The BTIA combines EII and EAI technologies to facilitate business activity monitoring (BAM) as well. Hayes notes, "Typically, EAI technologies link applications together or pass events from one application to another and allow event monitoring. EII complements this by providing access, in a single instance, to the current state of business data. That data may reside in multiple, distributed stores, but the user sees it as one coherent view, thus facilitating analysis."
What about data analysis and access, though? You know, slicing and dicing and all that? The BTIA pairs best-of-breed BI technologies with EII to facilitate complex data analysis. According to Business Objects' Cunningham, BI is the technology that enables users to successfully leverage all the data presented by EII. "Data integration technology can help you map a strategy and wire the underlying infrastructure together," he says. "However, BI technology helps organizations make sense of all that information. In other words, BI enables organizations to leverage the data they can now access and use it for their benefit," notes Cunningham.
Top-notch BI tools also enable the BTIA to provide users with proactive alerting to address problems before they balloon into disasters. To illustrate the power of the BI component in the BTIA, Cunningham describes a typical scenario where proactive alerting can affect the bottom line.
He describes a scene in which a food distributor must have a particular load volume on delivery trucks to meet margins, yet a good customer requires an emergency load. "In that business, margins are low, so if a truck leaves the loading facility partially loaded, the margin for the day is lost," says Cunningham. "In this scenario, the organization's dashboard would send the distribution manager a flag indicating that the load is too small. The manager would then have the information to decide whether or not to send the truck out anyway." The implications are clear. The customer may be so important that the act of overriding the margin is warranted, or maybe not. The point is that on a day-to-day, situational basis, management has access to real-time, proactive information to be able to make decisions that affect the bottom line.
With all the power of the BTIA, are there any limitations? The short answer is yes. As with any burgeoning technology, there are some limitations on data transfer and analysis. However, the future for EII, the foundational enabling technology behind the BTIA, is bright indeed.
In terms of querying and manipulating data, EII enables users to combine relational, XML, spreadsheet and content repository data and manipulate that data as if it were in a relational database. However, from an update perspective - bidirectional transfer in other words - most EII products only support updates to relational data. Says IBM's Hayes, "At this time, bidirectional data transfer is limited to relational data. It doesn't support insert, update or delete functionality to non-relational back-end data. That's for the future."
Most EII technologies also require help for multidimensional data analysis. "No one is going to try to do multidimensional analysis over a federated set of data stores," says Hayes. "The amount of data that would be required to move around would prohibit that right now from a performance perspective." IBM is looking at enhancements for its embedded caching technology to enable them to approach multidimensional analysis. Another solution would be to take the integrated data and move into a multidimensional cube created by a top-notch BI tool, which is already part of the BTIA anyway.
Despite its present and temporary limitations, building the business-time information architecture described will provide organizations with a tremendous set of benefits. The BTIA enables users to have access to integrated information from across the organization as well as the ability to compare that information regardless of context or content type.
The BI component of the BTIA enables powerful data analysis and provides for proactive alerting so that decision-makers have the right information at the right time and in a common, enterprise-wide view that provides one single version of the truth to the entire organization. In short, the BTIA, by leveraging EII and BI technologies, provides a solid architectural foundation that gives organizations the ability to begin solving their intractable information and analysis problems and get back to what they do best - growing the business for the future.
All information provided is of a general nature and is not intended to address the circumstances of any particular individual or entity. Although we endeavor to provide accurate and timely information, there can be no guarantee that such information is accurate as of the date it is received or that it will continue to be accurate in the future. No one should act upon such information without appropriate professional advice after a thorough examination of the particular situation. The views and opinions are those of the author and do not necessarily represent the views and opinions of BearingPoint, Inc..
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access