When we first described the major business intelligence, process and integration trends in 2007, businesses were flush with capital, CIOs were focused on strategic enterprise initiatives, business and IT leadership were aligning initiatives, and the establishment of BI and integration competency centers was all the rage. No one could have predicted the global recession's impact on BI, process and integration trends over the past months. Therefore, as we review and update the current state of BI, process and integration, we will focus on near-term trends because this recession is unlike any others we have experienced.

Since 2007, many organizations have had varying degrees of success in creating a BI foundation and aligning BI investments to corporate and IT performance management objectives. These accomplishments demonstrated the value of BI to improve enterprise visibility, drive business performance and enhance responsiveness to market conditions. We believe this has enabled businesses to more quickly and decisively adapt and react to global market changes.

Key Trends in BI

Overall, IT is maturing in establishing core and fundamental BI capabilities. Positive BI trends point to this as necessary to be successful. The IT organization will continue to:

  • Enhance BI project management processes to reduce risk and budget overruns,
  • Consolidate the BI application footprint to reduce application, data and support costs, and
  • Closely partner with business leadership to define and justify BI projects.
  • Given the economy, there are some fundamental trends regardless of industry.
  • Funding is limited. Businesses will approach IT investments differently as funding is tied to company cash flow as well as market conditions.
  • Capital investments are difficult to obtain and, in many cases, require approval from the board of directors. Defining compelling business value is a must.
  • Investment funding is tied to stock or market performance so businesses can better manage cash flow. Deferring, shelving or canceling projects is common.
  • Projects have been broken into smaller, incremental efforts that are funded within fiscal periods or quarters to closely manage capital and validate business value.
  • "Quick hits" are gaining emphasis as business teams are looking to get information sooner and may not wish IT to run everything through a data warehouse for key business insights. A challenge is that they may lose attention on getting solutions backfilled in the proper architecture at a later time.

The dominating BI trend for 2009 is that CFOs want to improve financial planning and analytics. An online survey by the BPM Forum reveals that 33 percent of companies replanned, reforecast or created what-if scenarios three or more times last quarter, which is more than double the percentage (16 percent) who reported doing so in last quarter's poll. Seventy-one percent of respondents expect to replan, reforecast and create what-if analyses more frequently going forward, versus 58 percent last quarter. For midsized and larger companies it climbs to 85 percent, versus 52 percent last quarter.
BI vendors are responding to this trend in two ways. The first is bundling of BI tools and capabilities as part of business upgrades to enterprise resource planning and supply chain management systems to improve cost control and business visibility. The second trend is the creation of FP&A packaged solutions to help clients cost-effectively and more quickly leverage their investments. BI vendors that are unable to provide this capability will face a competitive disadvantage in a consolidating marketplace.

There are continued internal challenges and complexities with BI implementations. Many businesses that have invested in the BI journey over the past two years have uncovered internal challenges and complexities associated with enterprise-level BI. Many of these same issues were discovered as part of enterprise application integration activities - the need for enterprise data governance, data architecture and data management. CIOs and their enterprise architects have made positive progress in collaborating with their business counterparts. Hence, they have baseline hard numbers on the business cost and inefficiencies related to poor data management and ownership. Although common issues are identified in the BI journey and EAI activities, the challenge is how to solve urgent needs now without trying to boil the ocean and stall momentum against enterprise efforts.

Large vendors are acquiring tools and platforms that offer the client a one-stop shop for BI. For example, the acquisition of Business Objects enables SAP to offer Business Objects as an integration and reporting platform. Oracle's continued acquisitions, such as Hyperion, Siebel and Sunopsis do no less. Microsoft also continues to offer a lower-cost, one-stop option. The challenge for these firms is to meld the best of the acquisitions and carefully articulate an adoptable product integration roadmap.

Integration Trends

Data integration, the binding glue of enterprise information architecture, is making strides into merging with other aspects of the enterprise architecture. As the demand for actionable information is melded with operational data to support business decisions, it is becoming more common that data latency is driving integration efforts to respond to real-time needs. In addition, the need for shared, consistent and high quality data is the driving need for data governance. Hence, master data management, extract, transform and load/data integration tools and the disciplines that support common data integration are moving closer to critical operations. As expected, organizations still face challenges when dealing with data integration.

There is often a lack of a meaningful and leveragable enterprise data model. A predominant need exists to architect the information data model in a fashion that is consistent, easy to use and fully exploitable for multiple enterprise purposes - whether to support an integrated data hub, operational data store or enterprise data warehouse. An adaptation of the various modeling styles can produce a model suitable for the broader purpose. For example, it is generally assumed that star schema models are well-suited for data marts, third normal form models are for enterprise warehouses, and deeper normalized models support ODSs and data hubs. However, a hybrid model, which is a mix of normalization techniques, may be perfectly suitable for a data hub, EDW or ODS. This purposed model tags entities as to their meaning and usage and its design is supported by a consistent methodology. A notable style is the data vault, a term coined by Dan Linstedt, or another form called anchoring modeling. These styles foster a consistent blueprint of maintaining a model, which naturally supports the requirement of uniform integration, ease of use and extensibility. This type of information architecture style can support the service-oriented architecture and BI process management integration needs.

Data integration standards are plentiful, but not enforced. Most IT organizations have shelves full of standards, yet according to Gartner, a mere 33 percent of surveyed organizations indicated they have enterprise standards for data integration and effectively enforce them. The remaining 67 percent may or may not have standards and do not enforce those at a consistent level. For purposes of reducing costs and maximizing investments in tools, those firms that work to mature their standards, governance and enforcement will substantially benefit from the extra attention given to data integration.

To integrate or federate? This continues to be a debate for a variety of reasons. Continued political and data ownership issues tend to drive these decisions rather than business needs and integration best practices. This often leads to inefficient and ineffective data integration solutions. The adoption of effective data governance, despite the type of architecture adopted, can substantially address the challenges of data integration. Policies and governance become more crucial in a federated architecture because data may be handled multiple times when transported, manipulated and stored in disparate and physically separate structures.

Elements of data integration are treated as a commodity. A notable trend is that outsourcing of data integration code development continues to increase. Since data integration development practices are mature, a significant number of large-scale BI development efforts are continuing to shift work offshore. Seasoned architects who typically shape the integration framework continue to remain onshore and maintain contact with offshore resources to ensure compliance and quality. Implementing a sustainable and trusted data integration architecture continues to be a complex problem and will continue to require highly skilled architects who are familiar with the business and the solution domain.

Enterprise architecture is moving toward extra-enterprise architecture. Due to the global nature of economies and businesses, information sharing is becoming more pervasive as firms share business data with their partners, government agencies or independent firms. The challenge is to produce a top-down or reference architecture that supports this cross-functional demand for data integration. Protocols that support data sharing such as Web and SOA services and other protocols, like Business Process Execution Language, are influencing how data flows are organized for ease of access and integration.

Data integration as a shared service. As companies explore avenues to streamline and reduce the cost of data integration across the enterprise, centralizing tools and skills is an option with a tangible benefit. Implementing a shared service model for data integration allows the business to put into place standard reference architectures, leveragable infrastructure and data integration standards applying skilled resources to speed solution development and boost quality. There are other aspects of BI where the shared service concept is making inroads, including BI reporting and delivery and BI application support.

Tactical MDM solutions should be aligned with strategic enterprise needs. The preferred trend is for organizations to adopt enterprise MDM. However, for the sake of time and cost, MDM solutions are being implemented for tactical purposes. These solutions are not fully integrated at the enterprise level and may support only a few business needs. Compromises are made for the short run, which conflicts with the spirit of enterprise integration since MDM typically is the core data service supported by SOA and BI process management services. The good news is that MDM adoption rates are up.

Data governance drives the need for quality data integration practices. Data governance is rapidly influencing the data integration process by defining the blueprint of the data structure; instilling transformation and conforming rules within data movement processes; defining the usage and integrity rules of certain data; and tagging data with ownership, meaning and lineage metadata. The key to data integration is properly gathering and understanding integration requirements as set by the policies and guidelines established by data governance.

Process Trends

Many IT organizations have made significant progress in their internal efforts to deliver enterprise BI services and refine their enterprise data architectures. As part of these enterprise initiatives, IT organizations have had the opportunity to introduce more formal and rigorous program and project management standards around delivery activities as well as engage business leadership regarding the business value of data governance.

IT organizations have been very quick to adopt and leverage IT portfolio management over the past two years. Doing so enables the CIO to engage business regarding needs, priorities, expected benefits, and capital investments. Likewise, it has enabled the CIO leadership teams and enterprise architects to have a more consistent dialogue with business leadership regarding tactical and strategic business needs, strategy and vision. These benefits come with the need for consistent tenacity and discipline in managing the relationship and rigors that define the portfolio.

As IT leadership has driven portfolio management to engage the business in establishing project priorities, funding and participation, they recognize the need for dedicated project management resources. This has been an industry-wide trend as businesses require (if not desire) managers to formalize their project delivery skills. Likewise, many IT organizations have created career paths for project management based on industry experience, project value and project management certification. IT leadership recognizes the fact that credible project managers foster better and deeper relationships with the business. Disciplined project managers also improve the consistency of how projects are delivered to the business while providing the rigors and discipline necessary to create a quality feedback loop back into the IT project portfolio. A challenge is to not disband the unique approaches to delivering BI solutions that make those projects successful.

IT leaders have been driving hard to achieve greater scales of economy and savings by consolidating and standardizing applications across the enterprise. As part of that consolidation, the business consumers are demanding more frequent and timely access to business data to develop new services, improve supply chain performance (e.g., business process management) or increase visibility into business performance (e.g., BI).

This has driven the need and understanding of why data governance is important. This is seen by the increased frequency and volume of data governance articles, publications and training. Data governance is an essential component to any BI and data management function; it provides the policies, practices and processes regarding how data is governed.

Data governance education is steadily growing and taking root in many organizations. Slow yet steady effort is being applied by CIOs, CTOs and enterprise architects as they engage and educate the business. For enlightened organizations, a champion for change is an executive business sponsor who has creditability with his/her peers.

The current state of the economy has and will continue to shape BI, data integration and process trends through 2010. Senior leadership has learned over the past few years the purpose of BI and how it improves business visibility. One can expect CFOs to drive the demand and provide the capital for BI in the area of financial planning, forecasting and analytics. Likewise, BI vendors are responding to this trend by creating prepackaged financial planning packages as well as bundling BI services and/or packages into every ERP or SCM offering and upgrade they make to their customers.

To support this continuing demand for more comprehensive BI, CIOs and their business counterparts are currently confronting the need to refine project management disciplines while establishing an enterprise data architecture and governance organization. Finally, IT is looking within itself to streamline their processes and technology assets to reduce cost and boost efficiencies. Data integration and governance processes have moved to the forefront of CIO strategic priorities (and challenges) as they address the need for strong business partnerships, developing enterprise data governance process, and establishing consistent perspectives and disciplines in the creation, control, distribution and management of enterprise data.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access