Macroeconomic Conditions and the Global Economy
If we look around us, it is pretty clear that the day and age of the global economy has arrived. Businesses are looking to transcend international and trade barriers in search of new markets for their products and services. Economies of scale are drawing businesses into the global business ecosystem due to cheaper raw materials, more cost-effective resources and new business partners. However, to really thrive in this global economy, businesses have to face a whole new set of macroeconomic conditions such as increasing regulatory compliance, stiffer competition, a variety of consumer channels demanding higher levels of service, shrinking product lifecycles and faster time-to-value resulting from mergers and acquisitions. Overcoming the challenges presented by these conditions requires the modern enterprise to continuously improve business agility in order to achieve its corporate goals.
Enterprises are realizing that access to critical business information at the right time is crucial for maintaining competitive advantage. However, the ongoing explosion and fragmentation of data across the enterprise and beyond complicates the situation. For example, without the availability of accurate, consistent and timely information, businesses cannot effectively respond to customer service demands, consistently rollout differentiating products, support inventory optimization efforts, rapidly on-board new systems or adhere to changes in information-centric regulations. To support such dynamic environments, enterprises are striving to evolve into what are called real-time enterprises by leveraging the power of their enterprise information assets. In this article, we will discuss how an IT organization can enable the real-time enterprise to improve business agility by delivering accurate, consistent and timely information across the extended enterprise.
The Real-Time Enterprise Enables Business Agility
A real-time enterprise uses business processes, IT solutions and best practices to ensure that information about its customers, products and partners is always up to date and of the highest quality. Armed with such information, a business can address increasing customer demands, changing market conditions and approaching competitive threats in a more cost-effective, expedient and proactive manner.
In the words of David Stodder, vice president and research director of Ventana Research, Businesses are being driven by growing customer expectations, intense competitive pressures, and constantly changing macroeconomic issues to become more adaptive, agile and efficient. To thrive in this environment, leading businesses are actively trying to evolve into the real-time enterprise an enterprise that maximizes the value of its information assets by integrating and analyzing critical data in its various front-office and back-office systems in real time and using it to drive more efficiency and profitability.1
If we were to make a list of the typical business processes that a modern business engages in to drive increased revenue or market share, we would notice a common theme across all of them. Lets consider a few examples, such as cross-selling and up-selling initiatives, point-of-sale fraud detection, real-time insurance policy quote generation, regulatory compliance, personalized best offers for marketing, supply chain optimization, retail out-of-stock replenishment, straight-through processing and reconciliation and 24x7 global operations. All of these business processes reflect the increased responsiveness involved in conducting business in this new global economy and mandate real-time or near real-time information. An effective IT organization can thus help a business make the leap to a real-time enterprise by enabling the availability of timely information to improve decision-making and operational efficiency.
The Information Latency Continuum and IT Projects
The timeliness of information, or what we refer to as the information latency continuum, can span a wide range, depending on how fresh the information must be to provide the most business value. The latency of information used for analytical purposes can typically range from weeks to days, as is required for business intelligence (BI) reports based on historical data. On the other hand, information used in operational scenarios such as inventory management, straight-through processing, customer on-boarding and online order confirmation typically must be delivered within hours, minutes or seconds.
As Ted Friedman, vice president and distinguished analyst, Gartner, mentioned in Survey on Data Integration Practices, Shows Move Toward Strategic Initiatives published June 6, 2008, "Most important for organizations to recognize is that their data integration will require a mix of latencies while real-time activity is on the increase, there will always be a need for higher-latency data integration work, since not all data in the architecture changes frequently, and not all processes, teams and roles are capable of harnessing real-time data."2
As we can see from the information latency continuum in Figure 1, different business scenarios have specific tolerance levels for receiving analytical and operational information. For example, the tolerance level for receiving order confirmation information after placing online orders on e-commerce Web sites cannot be greater than a few minutes, whereas it may be quite acceptable to send an inventory restocking notification to a supplier within hours. More exacting scenarios such as online self-service banking portals can drive the demand for more current or live customer account information to seconds.
In order to cost-effectively access and deliver timely and trusted data to support both analytical and operational scenarios, IT organizations typically undertake a variety of data integration projects. The success of a data integration project typically depends on the ability to meet an IT organizations service level agreements (SLAs) related to data latency, data completeness and data accuracy. To ensure the success of the data integration project, here are some questions to consider:
- Does the project support analytical or operational business purposes?
- Does the project require the movement of large data volumes or small data sets?
- Does the project require data to be delivered frequently or infrequently?
- Does the project need to ensure inaccurate data is not sent to downstream applications?
- Does the project need access to a variety of data sources and complex transformations?
Data integration projects thus need to effectively handle the integration of any type of data, structured or unstructured, guarantee accuracy and consistency and deliver information at the speed of the business. As we discuss in the next section, operational data integration can deliver significant competitive advantage by ensuring right-time information is available for a variety of business critical scenarios.
Operational Data Integration for Right-Time Information
Operational data integration describes a set of IT capabilities to access, cleanse, integrate and deliver any data, current and historical, at any latency for use in business processes related to operational execution and decision support. Data integration projects that involve operational data integration include real-time data warehouses, operational data hubs, data synchronization and data replication projects, and data services as part of an overarching SOA strategy.
Operational data integration projects require access to any data source at any latency (e.g., batch, change data capture) of any size to be integrated, cleansed of duplicates or errors, and delivered to any consumer channel in real time or near real time. Traditional approaches to operational data integration, such as enterprise application integration (EAI) have proven time-consuming, costly, and difficult to build and maintain. Furthermore, they dont address issues related to quality and data governance.
Operational data integration projects have varying requirements related to data volume, data latency, data quality and IT infrastructure sometimes employing different methodologies. To reduce costs, facilitate standardization and reuse, and improve time-to-value, a unified and flexible data integration platform should form the basis for an enterprise information strategy. Figure 2 provides a consolidated view of the typical data integration capabilities required across the various operational data integration projects.
Now that we have seen the variations of data integration capabilities across different operational data integration projects, lets examine each project and their relevant use cases.
Real-Time Data Warehouse
A real-time data warehouse typically employs a mix of batch and real-time data integration modes for processing large volumes of data. Real-time data warehouses provide information based on historical and current data for business processes related to operational execution and near real-time decision support. A real-time data warehouse enables a business to gain immediate insight into key indicators and metrics so it can quickly react and anticipate events or situations, thereby improving customer service and operational efficiency.
For example, in the fast-paced, ultracompetitive consumer retail market, accurate and up-to-date demand forecasting can make the difference between success and failure. Managers demand information at their fingertips to accurately communicate sales projections on a daily basis, manage retail stock replenishment hourly to optimize product presence on the retail shelves and collect point of scale (POS) data in real time to feed operational data stores (ODSs). A real-time data warehouse enables a right-time 360-degree view of key business areas such as sales, revenue recognition and demand planning.
Data replication projects involve the physical copying of data between systems at high speeds and with great reliability. Also, a data replication project must be able to support the continuous movement of data from one system to another as transactions occur in the source system. For example, a data replication project may copy relational database management system (RDBMS) tables from a source system to another RDBMS of the same or different type.
Data replication projects satisfy several types of business needs, such as regulatory compliance, real-time fraud detection, live reporting without disrupting business operations, zero-downtime migrations and disaster recovery. For example, a business needing to comply with regulatory constraints can leverage transactional data copied and archived for immediate or later analysis and compliance reporting.
Data synchronization projects are another type of project involving the copying of data. However, in data synchronization projects, the data copied may not be easily accessible and may require some transformation, conversion and additional information to fit the target system. Most data synchronization projects transfer data either one way or bidirectionally between one or more applications, while ensuring 24x7 availability and recoverability.
For example, a financial services company can improve the operational efficiency of its asset management services by eliminating the manual steps to enable straight-through processing (STP). With fast data capture and low-latency data synchronization between middle and back-office applications, the business can achieve lower operational costs and increased customer satisfaction.
Operational Data Hub
An operational data hub is used to share information consistently and accurately across applications to meet key business objectives. Real-time data integration hubs include data broadcasting, data consolidation, master data and transactional data sharing. Such kinds of projects often require the orchestration of data integration tasks to execute data routing rules, data integration acknowledgement and human interaction based on specific events (e.g., data changes and conflict resolution).
For example, in the ever-changing telecommunications industry, as part of a growth strategy, businesses may acquire related franchises and, as a result, diverse information systems. However, as time goes by, it is often observed that customer information is duplicated and distributed across many customer relationship management (CRM) databases for each geography and franchise. Having customer information distributed across many databases can inhibit the businesss ability to enhance and improve customer service and customer experience via their call centers. By building a customer data hub, the company can federate customer and service information, automate issue resolution and provide a single view of customers to call centers for better service. The customer data hub can also be used by marketing to better understand and manage customer behavior and help in cross-sell and up-sell opportunities.
Event-Driven Data Integration
Event-driven data integration is used to respond to business events as they arise or in anticipation of certain events before it is too late to react. In many cases, data changes are not predictable, resulting in data irregularities and inconsistencies. Nonintrusive direct access to data modifications at the moment it occurs enables immediate action to be taken either automatically or based on human interaction.
For example, in the retail industry, in order to ensure customer satisfaction, it is very important to continuously detect low stock levels and activate automatic restocking actions or alerts. By tracking inventory data in a timely manner, retailers can make decisions regarding products and prices more efficiently. Another advantage of real-time tracking is the monitoring of data quality issues to detect potentially irregular order treatment.
Real-Time Data Quality
The quality of data is extremely critical to businesses because they need to have confidence in the accuracy and consistency of the information that flows through their systems and to their partners. Businesses are thus investing in technologies to ensure the accuracy and consistency of the data they consume and share. Real-time data quality projects guarantee that as soon as data is created in a system, it is normalized and validated against standards either automatically or through human interaction.
For example, in the highly competitive financial services industry, service providers are constantly looking to enhance the customer service experience and support up-sell and cross-sell opportunities. In recent years, Web-based self-service banking portals are being offered as competitive differentiators that increase customer satisfaction, in turn driving greater revenue and market share. Real-time data quality projects proactively ensure that the customer account information that gets served up on demand is consistent and accurate across all financial products and all points of customer interaction (e.g,. online Web portals, call centers and regional branches).
Front-Office Data Automation
In the modern enterprise, employees typically use office documents and emails to communicate information that has to be integrated, mostly through retyping, into corporate information systems upon receipt. Today, most of this processing is still done manually. With front-office automation, it is now possible to process all types of documents and integrate the data into the information systems in real time.
For example, one market research provider regularly receives hundreds of office documents primarily through email attachments in spreadsheet format. In the pas, these documents were processed, corrected or rejected manually to make sure data could be integrated into their internal production systems, compiled and aggregated for market analysis. Front-office data automation enabled more efficient data processing by automatically converting the document to a structured format, analyzing the data quality and integrating the data into the production systems in real time, freeing up valuable human resources.
Back-Office Data Automation
Businesses have been increasingly adopting industry standards like SWIFT in the financial services industry, ACORD in insurance, and HL7 in health care to exchange information with their partners. These business processes require many forms of enterprise data (structured, unstructured or semi-structured) to be accessed, validated, integrated and delivered to users and applications across the company and between partners exactly when needed, be it batch, near real time or real time. Well-orchestrated back-office automation can reduce costs and improve operational efficiency.
For example, in the health care industry, there are significant inefficiencies associated with delayed or inaccurately paid claims and the failure to comply with regulations such as HIPAA. Back-office automation projects can utilize data integration technologies to reduce IT development and maintenance costs, accelerate compliance with pre-built HIPAA libraries and provide end-to-end visibility into claims processing.
The most responsive and agile businesses, or what we call the real-time enterprise, perform the best among their peers in todays constantly changing global economy. The real-time enterprise demands timely, trusted information to add value to products, customer relationships and business partnerships. This requires a flexible and scalable data integration platform that can handle all the complexities of analytical and operational data integration to guarantee the delivery of accurate and consistent information at the right time.
- Dave Stodder. Operational Data Integration for the Real-Time Enterprise. Informatica, June 2008.
- Ted Friedman. Survey on Data Integration Practices, Shows Move Toward Strategic Initiatives. Gartner, June 6, 2008.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access