The intense appetite for meaningful right-time information at companies of all sizes now demands that business intelligence (BI) mature from organizational and departmental point solutions to enterprise assets. Enterprise demands, coupled with constrained IT budgets, further necessitate that BI information management (IM) align with legacy enterprise information management (EIM) services. As more is expected from BI services, enterprises must also change how they govern the processes that define, maintain and distribute that information.
Why is BI suddenly an essential element of the enterprise information framework? There are a multitude of business and IT considerations. First, CIO and IT executives are confronted with growing demand from CEOs and corporate boards for increased visibility. To improve bottom-line performance, CIOs are sponsoring a growing number of corporate performance management initiatives to:
- Address how an enterprise defines, introduces and manages nonfinancial performance metrics, including business-cost variability, innovation and customer satisfaction;
- Remove internal barriers related to how the business creates and aligns internal performance metrics to strategic goals and initiatives;
- Demonstrate that IT innovations can help improve business visibility and growth;
- Establish BI and data governance boards to maintain executive and cross-organizational sponsorship and to drive the data integration and process changes necessary to realize the business objectives; and
- Align business strategy and goals to justify IT investments and initiatives.
For the CIO, internal organizational challenges are no less complex. IT organizations are recognizing the BI shift to the enterprise and that they need to continue their development of BI competencies, processes and governance. In addition to strategic drivers, IT is faced with tactical needs to address historical efforts that:
- Failed, or costly implementations that delivered too little too late;
- Created point solutions that cannot be integrated or scaled;
- Reduced total cost of ownership resulting from redundant or silo implementations;
- Resolved fundamental enterprise data quality and integration complexity issues;
- Refined business, IT and project delivery processes to yield high quality solutions; and
- Insufficient processes to address increasing frequency of data updates.
Market factors are increasingly influencing IM direction, ultimately driving CIO decisions. Major enterprise applications such as supply chain management (SCM), enterprise resource planning (ERP) and customer relationship management (CRM) include some form of data integration, enterprise data warehouse, reporting and analytical applications and services, embedded in their solutions. CIOs are often compelled to support these applications, in some cases putting up with suboptimal capabilities because many of the package components are in varying states of maturity or capability. However, open source and BI commoditization are making BI services more affordable.
In addition, market consolidation is yielding a greater range of global BI services and capabilities for enterprise one-stop shopping. Market commoditization is driving down cost, enabling CIOs to scale to the enterprise in a controlled and cost-effective manner. Limited supply of enterprise BI expertise within the U.S. and Europe is requiring businesses to more frequently collaborate with third parties for BI lifecycle support.
Key elements, which BI IM share with EIM, are data integration and process. Data integration is the technology enabler, while process addresses the authority and management of the information asset.
The success of enterprise data integration hinges upon leveraging a mature information architecture and integration practices prevalent in BI, while evolving to account for recent architectural concepts, such as service-oriented architecture (SOA), to ensure the hunger for timely and meaningful information is satisfied. BI has a lengthy history of addressing data quality improvement and enrichment, master and reference data conformity, the capture and exposure of metadata while efficiently moving and storing large volumes of information. However, in terms of EIM, the challenge today faced by BI data integration is aligning its architecture with the EIM reference architecture, which treats BI as a data service. Even though BI practices and teams are mature, the architect must contend with a set of serious challenges and issues.
Address the lack of a global data model. EIM demands a comprehensive enterprise information model that reflects both BI and operational business processes. The good news is, the mature data warehouse model is generally reflective of the enterprise view of business information. The adoption of more abstract and industry-specific models may benefit the development of the enterprise model.
Debate of using BI as an operational data store (ODS). A key point is the discussion regarding the method of representing integrated data in either a virtual or persisted view. This is influenced by capabilities of the data integration approach, whether extract, transform and load, enterprise application integration, enterprise information integration or SOA. An existing ODS may be an attractive option because many of the integration features (data quality, transformation, integration) are imbedded in that framework.
Define how and where to implement master/metadata; data cleansing, business/transformation rules. Master data management (MDM) is sensibly migrating to the enterprise level because business processes are demanding more accurate and consistent master data. MDM vendors are converging in terms of domain, scope and industry focus due to the importance of MDM to SOA and to your business.1
Metadata is pervasive in the BI process, yet often poorly integrated and managed. As BI IM folds into EIM, it becomes more imperative that metadata definition is effectively coordinated with EIM governances. The business challenge is cataloging and classifying the information asset. The EIM architect must determine the appropriate integration mechanism to maintain and expose this information.
Determining the placement of data cleansing and transformation processing is dependent upon information latency requirements and complexity of the business transformation rules. Most integration tools are evolving to support both offline and real-time transformation processing.
Lack of data quality management within legacy and in-house applications. Historically, the business has delegated the task of presenting good information to BI applications. Hence, BI teams effectively cleaned up the mess housed in operational systems yet could not enforce quality in operational applications. Quality management is a cornerstone to EIM, and mining BI-pioneered techniques will only help to serve this undertaking.
Define an enterprise data integration architecture. In most organizations, operational applications are nonintegrated silos of data. Enterprise information architects have struggled mightily to bring order to this chaos, with little success. Primarily, the business did not have the stomach for long implementations and high cost. BI has long succeeded with data integration because the business has demanded it support strategic analytic and reporting needs. EIM has recently gained momentum since the business is realizing that operational data must integrate to support critical business processes. SOA, BI and vendor communities must come together to develop standards and approaches for EIM architecture.
Acknowledge technology disruption from Web 2.0/SOA. BI and SOA architecture disciplines support similar needs: IM, metadata, data integration and data quality. There is an education process that must occur between the technology camps (and integration vendors) to ensure each understand and exploit the synergies of both architectures. At this time, tools and opinions abound, and the right choices must be rationalized against business values.
When selecting the right long-term partnership with vendors, one must look for the right mix of capabilities. The market is growing due to demand and undergoing significant change, as recognized by Gartner. The data integration tools market will grow to $2.6 billion by 2011. This will reflect a strong five-year compound annual growth rate of 17.3 percent as organizations continue to invest in technologies critical to managing data assets.2
You must separate hype from proven product features and seek out a vendor that can offer the broadest set of real integration capabilities. Vendors are working toward offering a more complete package that addresses information integration needs. Vendor acquisitions will abound; however, at this time, it is tough to gauge the best partner for a long-term relationship.
In bringing BI to the enterprise, many struggle with the challenge of how to manage this transition, particularly when it occurs at multiple levels within the business and IT. Very few groups and applications are shielded from this change. It affects how people work, how systems exchange information, how systems are introduced and integrated into the enterprise architecture and how data quality and process issues are resolved at their source.
This is the moment of change. CIOs have the opportunity to unite the cross-functional business groups. They must all contribute to the solution. Groups quickly realize that their independent efforts are creating data quality and data consistency issues. Thus, driving the need to create data governance organizations, competencies and processes are necessary to achieve and sustain the required visibility and disciplines.
As written in many recent data governance articles published in DM Review, governance is very hard work that requires dedication over a long period. Enterprise-level governance is new and requires business and IT silos to be removed. This reality is reaffirmed by a Gartner study that predicted by 2008, less than 10 percent of organizations will succeed in their first attempts at data governance.3 This is pretty depressing news for someone who has been chartered with this task.
To be part of the successful 10 percent, one needs to follow basic guidelines:
- Obtain executive sponsorship early;
- Start small to build momentum, focus on critical projects with clear goals;
- Maintain and grow business commitment;
- Create effective oversight, data ownership and data stewardship with the business; and
- Introduce effective data quality processes and supporting quality metrics.
Introducing a competency and discipline into a corporate culture is a journey; thus, persistence and patience are required. When introducing processes to govern how information is created, distributed and used, one needs to educate people on the need and value of process governance as well as understanding how people think, what drives their behavior and how much change they can absorb. There are some suggestions on how to improve your probability of success:
- Create and grow a business-awareness campaign regarding governance and its impact on the organization. Have your executive sponsor help you educate the knowledge workers on how employee and business actions affect the enterprise and the need for change to improve business visibility. This reduces fear, uncertainty and doubt.
- Identify and educate key business and technology stakeholders on the value of BI and data governance. Engage them early to define the scope and governance charter. You will need knowledgeable and empowered political influencers to enable your success beyond an executive sponsor.
- Work early with your executive sponsors and political influencers on how to reward good behavior and address and correct poor behavior. People need positive reinforcement as process and governance require people to change and to follow someone else's rules.
- Be consistent and disciplined in how your governance and process teams work by following the rules you implement. Otherwise, you might confuse people and be perceived as empire building through process "gaming."
- Listen closely and embrace those who are resistant to the change. They may have experience or information on past efforts and could potentially influence your work. Conversely, politically charged environments require you to "keep your friends close and enemies closer."4
In summary, there is a perfect storm forming within the enterprise today. From the business side, executive leadership is demanding greater visibility into internal performance metrics that are consistent, predictable and trustworthy. In contrast, IT organizations and budgets are stretching to scale BI to the enterprise while having to address very visible data quality and process issues.
To merge the divergent paths on this journey, there will be increasing demands on unifying BI with data integration and governance at the enterprise level. This unification will require business and technology leaders to build new competencies, disciplines and perspectives on how they create, control, distribute and manage data as a valued enterprise asset.
- Andrew White, John Radcliffe. "Vendors Have Different Approaches to Implementing Master Data Management." Gartner Inc., 16 November 2006.
- Colleen Graham, Horiuchi Hideaki, Dan Sommer and Bhavish Sood. "Forecast: Data Integration Tools, Worldwide, 2006-2011." Gartner Inc., 4 May 2007.
- Hannah Smalltree. "Data governance requires checks and balances, Gartner says." SearchDataManagement.com, 17 November 2006.
- John Ladley. "Get 'er Done - Executing the Architecture." DMReview.com, 27 July 2006.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access