- Constant business change necessitating rapid IT response that includes changing information requirement specifications;
- Growing data volumes and complexity that increase business risk and reduce agility; and
- Financial constraints necessitating cost-effective IT solutions that meet business objectives and not simply “technology for technology’s sake.”
Traditional approaches such as data consolidation and replication alone have not kept pace. As a result, federated data models have evolved to complement these investments and fill the gap.
Data Federation at a Glance
Federated approaches to data integration, also called data virtualization, take an integration approach that combines data from multiple, disparate sources – anywhere across the extended enterprise – in a unified, logically virtualized manner for consumption by an array of transactional and business intelligence applications.
Frequently deployed at two levels, data federation typically complements existing data integration methods such as consolidation and replication. At the project level, data federation virtually integrates the data required in support of a specific application or use case. On an enterprise level, it is typically implemented as common services or as a loosely coupled data abstraction layer to share data from multiple sources across multiple applications and uses.
Common Data Federation Usage Models
IT teams across a diverse set of industries including financial services, manufacturing, energy, telecommunications, media/entertainment, pharmaceutical, health care and more have deployed a federated approach to data integration at both the project and the enterprise levels. The aggregated experiences of these enterprises may be summarized and characterized by five common data federation usage models. Enterprise architects and data processionals seeking to integrate disparately located and sourced data more rapidly and efficiently should consider applying one or more of the following five models to their own information systems:
- Project-level data federation,
- Data warehouse extension,
- Enterprise data sharing,
- Real-time enterprise data infrastructure, and
- Cloud data integration.
Project-Level Data Federation
Project architects typically begin data federation at a project level, after evaluating a number of business, data source and data consumer considerations. These include, but are not limited to:
- Time-to-solution: How rapidly does the business need a solution? For rapid turnarounds, data federation has the edge. For solutions requiring significant dimensioning of data, physical consolidation is the stronger, albeit longer-to-deliver choice.
- Resource allocation: What is the size of the budget? How big is the staff available to support the creation and maintenance of the data infrastructure? For projects without sizable budgets, data virtualization offers a high ROI and typically requires fewer staff members to deploy and support.
- Risk tolerance: What is the potential for building a solution that meets current data integration specifications, but isn’t sufficiently agile to meet unexpected changes that are likely to occur soon after deployment? For projects where little change is anticipated and the risk tolerance levels are high, physical consolidation data integration may be the preferred choice. For enterprises where data sources and information requirements change frequently and the risk tolerance is low, federated data integration is the best choice because it offers the most agility.
By taking a federated approach to data integration, architects and developers have a flexible palette of low-cost, rapid-deployment integration options including: federated views, data services, data mashups, in-memory and database caches, virtual data marts and virtual operational data stores.
Data Warehouse Extension
Supporting critical yet ever-changing information requirements in an environment of ever-increasing data volumes and complexity has and will continue to drive the demand for enterprise data warehouse-centric solutions. However, business change has begun to outpace EDW evolution, with significant volumes of enterprise data residing outside the EDW, due to increasingly distributed business value chains, greater workforce mobility, cloud computing and more.
Data federation preserves and extends EDWs through a range of flexible data integration techniques including:
- Integration with external, intra-day or detailed data,
- MDM hub extension – 360-degree view,
- Data warehouses federation,
- Hub and virtual spoke,
- Enterprise architecture,
- ETL sources extension,
- Data warehouse prototyping, and
- Data warehouse migration.
By integrating information from outside data sources, data federation avoids the costs, potential risks and time required to modify and support ETL and data warehouse structures.
Enterprise Data Sharing
Enterprise data sharing is becoming increasingly popular in enterprises and government agencies where various teams share an assortment of analysis and reporting tools to access and analyze large amounts of diverse data across disparate sources and multiple geographical locations.
To enable enterprise data sharing and overcome the complexities inherent in multiple geographies and data sources, federated data integration leverages service-oriented architecture principles including abstraction, decoupling of data from its sources, reuse and more, while also supporting a range of internal and industry data standards.
