BI, Process and Integration Trends

Information Management Magazine, August 2007

Craig Izydor, Patrick McCollum

The intense appetite for meaningful right-time information at companies of all sizes now demands that business intelligence (BI) mature from organizational and departmental point solutions to enterprise assets. Enterprise demands, coupled with constrained IT budgets, further necessitate that BI information management (IM) align with legacy enterprise information management (EIM) services. As more is expected from BI services, enterprises must also change how they govern the processes that define, maintain and distribute that information.

BI Trends

Why is BI suddenly an essential element of the enterprise information framework? There are a multitude of business and IT considerations. First, CIO and IT executives are confronted with growing demand from CEOs and corporate boards for increased visibility. To improve bottom-line performance, CIOs are sponsoring a growing number of corporate performance management initiatives to:

• Address how an enterprise defines, introduces and manages nonfinancial performance metrics, including business-cost variability, innovation and customer satisfaction;
• Remove internal barriers related to how the business creates and aligns internal performance metrics to strategic goals and initiatives;
• Demonstrate that IT innovations can help improve business visibility and growth;
• Establish BI and data governance boards to maintain executive and cross-organizational sponsorship and to drive the data integration and process changes necessary to realize the business objectives; and
• Align business strategy and goals to justify IT investments and initiatives.

For the CIO, internal organizational challenges are no less complex. IT organizations are recognizing the BI shift to the enterprise and that they need to continue their development of BI competencies, processes and governance. In addition to strategic drivers, IT is faced with tactical needs to address historical efforts that:

• Failed, or costly implementations that delivered too little too late;
• Created point solutions that cannot be integrated or scaled;
• Reduced total cost of ownership resulting from redundant or silo implementations;
• Resolved fundamental enterprise data quality and integration complexity issues;
• Refined business, IT and project delivery processes to yield high quality solutions; and
• Insufficient processes to address increasing frequency of data updates.

Market factors are increasingly influencing IM direction, ultimately driving CIO decisions. Major enterprise applications such as supply chain management (SCM), enterprise resource planning (ERP) and customer relationship management (CRM) include some form of data integration, enterprise data warehouse, reporting and analytical applications and services, embedded in their solutions. CIOs are often compelled to support these applications, in some cases putting up with suboptimal capabilities because many of the package components are in varying states of maturity or capability. However, open source and BI commoditization are making BI services more affordable.

In addition, market consolidation is yielding a greater range of global BI services and capabilities for enterprise one-stop shopping. Market commoditization is driving down cost, enabling CIOs to scale to the enterprise in a controlled and cost-effective manner. Limited supply of enterprise BI expertise within the U.S. and Europe is requiring businesses to more frequently collaborate with third parties for BI lifecycle support.

Key elements, which BI IM share with EIM, are data integration and process. Data integration is the technology enabler, while process addresses the authority and management of the information asset.

Data Integration

The success of enterprise data integration hinges upon leveraging a mature information architecture and integration practices prevalent in BI, while evolving to account for recent architectural concepts, such as service-oriented architecture (SOA), to ensure the hunger for timely and meaningful information is satisfied. BI has a lengthy history of addressing data quality improvement and enrichment, master and reference data conformity, the capture and exposure of metadata while efficiently moving and storing large volumes of information. However, in terms of EIM, the challenge today faced by BI data integration is aligning its architecture with the EIM reference architecture, which treats BI as a data service. Even though BI practices and teams are mature, the architect must contend with a set of serious challenges and issues.

Challenges

Address the lack of a global data model. EIM demands a comprehensive enterprise information model that reflects both BI and operational business processes. The good news is, the mature data warehouse model is generally reflective of the enterprise view of business information. The adoption of more abstract and industry-specific models may benefit the development of the enterprise model.

Debate of using BI as an operational data store (ODS). A key point is the discussion regarding the method of representing integrated data in either a virtual or persisted view. This is influenced by capabilities of the data integration approach, whether extract, transform and load, enterprise application integration, enterprise information integration or SOA. An existing ODS may be an attractive option because many of the integration features (data quality, transformation, integration) are imbedded in that framework.

Define how and where to implement master/metadata; data cleansing, business/transformation rules. Master data management (MDM) is sensibly migrating to the enterprise level because business processes are demanding more accurate and consistent master data. MDM vendors are converging in terms of domain, scope and industry focus due to the importance of MDM to SOA and to your business.¹

Metadata is pervasive in the BI process, yet often poorly integrated and managed. As BI IM folds into EIM, it becomes more imperative that metadata definition is effectively coordinated with EIM governances. The business challenge is cataloging and classifying the information asset. The EIM architect must determine the appropriate integration mechanism to maintain and expose this information.

Determining the placement of data cleansing and transformation processing is dependent upon information latency requirements and complexity of the business transformation rules. Most integration tools are evolving to support both offline and real-time transformation processing.

Lack of data quality management within legacy and in-house applications. Historically, the business has delegated the task of presenting good information to BI applications. Hence, BI teams effectively cleaned up the mess housed in operational systems yet could not enforce quality in operational applications. Quality management is a cornerstone to EIM, and mining BI-pioneered techniques will only help to serve this undertaking.

Page 1 of 3.