The intelligent use of information assets is always important to enterprises, but perhaps now more so than ever, given today’s soft economy. Information assets often play a key role in day-to-day business transactions as shown in the following examples:
- To counter a new competitive threat, business analysts need data from a variety of sources, including the enterprise data warehouse, marketing and financial systems, software as a service customer relationship management system and outside research data providers.
- To adjust inventory levels to meet fluctuating demand, supply chain analysts need up-to-the-minute supply data from internal and external suppliers across the globe, as well as consolidated demand information from across multiple sales channels.
- To extend the life of older capital assets in order to delay new equipment purchases, plant maintenance teams need to compare failure and repair data across plants.
Data virtualization is a proven data integration technique that enterprises and government agencies use to leverage their information assets. Data virtualization adoption is on the upswing as these enterprises and agencies seek to complement their earlier data consolidation and data synchronization investments. Moreover, data virtualization usage has recently accelerated as its adoption moves beyond individual projects to enterprise-wide deployments.
Several patterns of enterprise-wide data virtualization have emerged to solve broad business problems in a more consistent way. One pattern – enterprise data sharing -- is increasingly popular where large numbers of information consumers use a range of analysis and reporting tools to view and analyze large amounts of diverse data, from disparate sources or multiple geographical locations.
Components of Enterprise Data Sharing
Three key components combine to enable enterprise data sharing: XML industry standards, data services and data virtualization middleware (See Figure 1.)
Industry-wide XML data standards establish a common, agreed-upon consumption format for all data consumers, regardless of source format or location. In recent years, these standards have become increasingly important as data is shared within myriad departments and groups of a single large enterprise, as well as beyond the firewall to partners, suppliers, customers and others.
These XML industry standards-based data abstractions are typically developed and deployed using data services. This service-oriented approach has several advantages. First, each service can be developed, deployed and modified as an independent, standalone unit, providing greater flexibility and reusability. Second, the data services loosely couple the data sources and consumers, and therefore reduce the impact of changes at either source or consumer level. Third, data services can be developed and changed easily and rapidly using modern drag-and-drop development tools, thereby providing the agility required in today’s fast-paced business world. Finally, they can leverage other data services to split the work, such as sourcing services, federation services and standards transformation services, to provide even greater flexibility and reuse.
Developers use data virtualization middleware to build and run the XML standards-based data services. Top-of-the-line data virtualization development tools are used to conform data from diverse sources to internal- or industry-standard formats. Ensuring a high-productivity development environment, these tools provide native support for popular languages, including SQL, XQuery and Java, as well as graphical development techniques, and automated code generation and performance optimization. Upon completion, these services are deployed from the development environment to the data virtualization server where at run time, consuming applications can call these data services on demand via protocols such as SOAP, JMS and REST to query the requested data. The high-performance data virtualization server then executes the data service, performing all the necessary accesses, queries, federations, abstractions and deliveries necessary to deliver the required data from the various sources to the consumer. Operational transparency is a key feature, enabling the data virtualization middleware to fit easily into any existing IT environment. Rapid time to solution, lower total cost of ownership, and effective leveraging of existing staff and technology resources are the primary benefits that data virtualization middleware delivers.
Barriers to Overcome
Although data industry pundits at Gartner and Forrester Research recommend data virtualization as a strategic tool in the data integration tool belt, there lingers a general, preconceived notion that an enterprise data warehouse is the only way to share data on a large scale. It goes without saying that enterprise data warehouses solve many problems. However, 15 years of industry experience show that only about 25 percent of users actually leverage their warehouses, often because only a fraction of available enterprise data, typically summarized financial, sales and marketing data, is housed within them. One way, therefore, to overcome this barrier is to position the enterprise data warehouse as a key source of sharable data, along with other sources such as operational data stores, source transaction systems, external sources and more.
Interestingly, XML-based industry standards are both a potential barrier and a significant driver. While standards such as MIMOSA for process manufacturers including oil refineries, PIDX for upstream exploration and production, and MIEM for maritime information are starting points to fulfilling the need for such standards comprehensive standards are still lacking across the board.
Finally, there is the question of data ownership, or, put another way, the outdated attitude that says, “It’s my data, so why should I share it with you?” Enterprise data sharing helps to counter this resistance by using agreed-to standards and common approaches. This emphasizes the enterprise, without imposing the onerous infrastructure of traditional consolidation or standards-compliance approaches.
