As enterprises scale their e-business systems to accommodate an ever- increasing number of transactions, they face a new challenge ­ and opportunity. As the data that results from these transactions reaches staggering volumes, it threatens to become a silo of information that derails long-term data integration efforts among legacy systems. Carefully built customer relationship strategies and integrated supply chains can't deliver expected results without integrating the e-channel. At the same time, data from e-business systems provides information at a more discrete level than ever before, presenting a tremendous opportunity to understand customers, fine-tune transaction processes and measure business performance. Clearly, capturing and leveraging the enormous influx of data from e-business operations will become key to enterprise success.

With the new economy dominating headlines, there is no escaping the startling e-business statistics: Gartner-Group predicts that B2B e-commerce will reach $7.3 trillion by 2004 and B2C e- commerce will reach $380 billion by 2003.1 Faced with numbers like these, no CEO can afford to ignore the enormous opportunity presented by e- business. Enterprises are getting the message loud and clear: build a solid e- business presence or risk being left behind.

Reflecting this imperative, enterprise spending with e-business service providers will skyrocket over the next several years. GartnerGroup predicts that spending on e-business professional services in the U.S. will reach $50.4 billion in 2003, up from $10 billion in 1999.2 The driver of this dramatic increase in spending is the need for deep systems and network integration. Until now, most enterprises have focused on designing a graphically pleasing front end for the customer. Now, however, enterprises want systems integrators that can help them cope with a rising number of transactions, tie together their front and back offices, and enable more sophisticated e-business strategies.

Influx of E- Business Data

As part of their integration efforts, enterprises need to deploy solutions that deal with the enormous influx of data from e-business operations. According to META Group, e-commerce is driving data storage requirements at 400 percent annual growth,3 and GartnerGroup expects the average online enterprise to be managing more than 120 terabytes of customer data by 2004.4

The clickstream is the main culprit in the explosion of e-business data, enabling enterprises to track the smallest of customer gestures on their Web sites. Many enterprises also plan to augment their own customer data with data from third-party Web sites. For some enterprises, the data doesn't just come from the customer side ­ B2B exchanges and B2C commerce hubs also need to aggregate a huge amount of data coming from the supplier side.

Now that most enterprises have established their e- business "storefronts" and enabled basic transactions, it's no longer possible to claim simple e-business capability as a competitive advantage. Enterprises must fight to attract and retain customers, and to keep up with rapid changes in technology and business models. The ability to successfully utilize data will be one factor in determining e-business winners.

Three components of e-business success are driving the need to capture and use data: building customer relationships, integrating channels and enabling partnerships.

Building Customer Relationships

As e-commerce sites proliferate, it is becoming more difficult for enterprises to attract and retain customers. Now that literally anyone can use the Web to sell products and services directly to customers, consistently drawing traffic to any one site is a challenge. Drawing purchasing traffic is even harder. As Web site clutter increases, each marketing dollar spent becomes less effective. GartnerGroup estimates that by 2003, enterprises will receive less than $0.37 return on every dollar spent to promote a Web site.5

Effective use of customer data provides one answer to this marketing dilemma. Through careful collection and analysis of clickstream and application logging data, enterprises can create an optimal purchasing experience for customers, encouraging repeat site visits. The applications for clickstream data range from "mass customization" of a Web site's appearance to identification of technical problems that are hindering customer purchases.

Customer self- service is another data-related opportunity presented by the Web. For example, customers of a telecommunications firm could access an active data warehouse or data mart to determine their account status, change their service options or register a service complaint. No matter what the application, intelligent usage of data helps an enterprise build a base of loyal e-business customers and meet their growing expectations for service quality.

Integrating Channels

As brick-and-mortar enterprises build their e-commerce systems, it will be important for them to tie data from their Internet operations to data from existing sales and service channels. A single view of the customer is crucial to building long-term relationships ­ few things are more frustrating to a customer than being unable to get consistent service across all of an enterprise's contact points. In terms of formulating business strategy, enterprises need a consistent view of performance across all channels and units. For example, is a company's Web site cannibalizing other sales channels; and if so, is the cannibalization helpful or harmful?

Not just brick-and-mortar enterprises will face this problem. Surprisingly, industry analysts predict that many pure Web players will turn themselves into click-and-mortar businesses, establishing a physical presence to complement their online efforts. In fact, GartnerGroup predicts that the winners in e- business, especially among those who have tangible product to sell, will be hybrid enterprises that combine the best of the Internet and physical worlds.6 Again, these hybrid companies will need to integrate customer and performance data across all channels in order to get a full picture of their operations.

Enabling Partnerships

Partnerships are the next wave in the e-business world, a natural outgrowth of a fiercely competitive environment. From B2B exchanges to B2C commerce hubs, many Web sites are bringing together large numbers of buyers and sellers to do business in an online marketplace. Considering the difficulty in drawing traffic to a site, this makes economic sense. The sites that act as intermediaries concentrate on attracting buyers, while the suppliers they bring together focus on delivering the products and services that they sell.

An important factor in the success of these online marketplaces is information sharing. Suppliers need to provide intermediaries with data about not only their products and services, but also about their capability to deliver those products and services. For example, intermediaries best serve buyers when they know, in real time, the inventory levels or expected delivery dates for suppliers' products. This requires suppliers to share some of their internal data with intermediaries.

In return, intermediaries take on the not- insignificant challenge of aggregating supplier data and presenting it in catalog format to buyers. Gartner-Group estimates that one update to an online catalog can cost as much as 50 percent of the initial implementation cost.7 Additionally, intermediaries need to gather selected buyer data and provide it to suppliers as an incentive to participate in the marketplace. Suppliers can gain valuable insight into their competitors' customers as well as their own through data provided by intermediaries.

Managing E-Business Data

E-business data can provide some exciting competitive advantages, but putting systems in place to manage and use the data can be extremely challenging. Following are descriptions of each step in the process and some considerations for ensuring successful deployment:

Capturing E-Business Data. The primary form of data generated by e-business is clickstream data. Clickstream data is defined as a time series of user interactions that can be assembled into an organized session of activities. This data can be gathered from a variety of sources, including a Web site's log files, partner Web sites, ISP activity and other third-party sources. Another form of e-business data comes from Web application logging, which is defined as the ability of an application to log information about products, services and displayed information. Data from application logging allows an enterprise to obtain a complete picture of a customer transaction, including the content displayed throughout a customer's visit to a Web site. When integrated, clickstream and Web application logging data allow an enterprise to capture the entire customer interaction experience and perform sophisticated analysis to optimize that experience. Enterprises must be careful, however, to choose the right technology solution so as not to drown in the flow of data. Forrester Research reports that many companies have been frustrated by simplistic tools that have severe performance and storage limitations ­ and may even compromise the integrity of the data.8 Clearly, enterprises need to look for scalability and high performance in their e-business data capture solutions.

Using Data Dynamically. Once e-business data is captured, enterprises will want to use it to enhance customer relationships through personalization. Personalization customizes the Web shopping experience to enable cross-selling, up-selling and segmented service and support opportunities. Ideally, personalization will result from data that encompasses a customer's entire shopping experience. The first component of this is data from a customer's current activities at a Web site, captured through real-time application logging. The second component is historical data, including information about a customer's past Web purchases and customer history gleaned from legacy applications. The third component is the customer's online behavior, which tracks the customer's activities as they surf the Web. The final component is syndicated data, which provides third- party insight into general customer needs and preferences. The primary challenge in using data dynamically is completing sophisticated analysis in real time. The personalization experience has to happen at the click of a mouse, calling for fast and reliable systems that can collect and analyze the necessary data instantaneously. Customer self-service over the Web presents another set of challenges involving database scalability and reliability. If an enterprise allows its customers to directly access an active data warehouse or data mart, it must ensure that the system is scalable enough to handle a rapid increase in the number of users. The system will also have to be extremely reliable in order to avoid service outages and able to operate 24x7 to accommodate customers around the globe.

Integrating Data from Many Channels. Enterprises must integrate data from all sales and service channels in order to get a complete picture of their operations. This involves integrating data from traditional channels (such as physical stores and print catalogs) with new Internet channels into one high-performance enterprise data warehouse. While this sounds like a straightforward process, pitfalls abound. First, enterprises must scale their systems to deal with the enormous mass of data gathered from disparate sources. Second, different channels might have different business systems, developed over time to meet the unique needs of each. Sorting out the kinds of data provided by each of these systems adds a layer of complexity to the integration process. Finally, after gathering data from disparate systems, enterprises will need to reconcile the various data formats, check for redundancy and clean the data.

Aggregating Data. Intermediaries who operate online marketplaces need to aggregate large amounts of supplier data into catalog format. This data must be updated regularly to incorporate price changes, sales promotions, and new suppliers and products. Given this need to make frequent updates and additions, enterprises need to correctly scale their data aggregation technology from the beginning. If the solution is not built to anticipate a growing number of suppliers and increasingly complex data, the enterprise will be forced to make constant and expensive upgrades to software and hardware.

Components of a Successful Data Solution

Wherever an enterprise is in its e- business evolution, one thing is clear: the technology it chooses will be a determinant of success or failure. Every technology solution for e-business data must incorporate several characteristics in order to be successful:

Scalability. Industry analysts agree that the influx of data from e-business will probably exceed even their own wildest expectations. According to META Group, e-business data extraction efforts will soon need to deal with 10 times more data than current best practices. Enterprises such as and eBay have already, in their relatively short life spans, exceeded data storage systems developed by banks over more than 10 years.9 Any data solution an enterprise selects needs to be flexible enough to accommodate rapid increases in data volume ­ potentially billions of clicks a day that must be integrated with other data sources. It must also extract, aggregate and transform data reliably across these many channels to accommodate increasingly complex and granular requests for user access. Scalability must be present in all solution layers ­ hardware, software and database ­ to deliver the massive throughput, large-scale storage, and flexible and reliable access to data that define truly scalable solutions. These solutions typically rely on parallel and scalable technology frameworks. Since they are durable enough to accommodate growth without requiring replatforming and additional software, they offer a low cost of ownership.

High Performance. Data solutions must be fast, reliable and able to run 24x7. The global nature of e-business and rising user access requirements demand solutions that can meet these requirements. Slowdowns and outages will result in lost data, lost knowledge and lost business.

Data Integration. Integrating data from e-business channels with data from existing systems is crucial to the success of customer relationship and supply chain management initiatives. Enterprises must not allow e-business data to become yet another silo of information, or they will undermine existing efforts to fully synergize all areas of the organization.

Real-Time Responsiveness. In the online world, decisions can't be made a week or even a day later. Data needs to be analyzed and business rules need to be applied immediately. Data solutions must be capable of processing and acting on the click of a mouse in real time. Without this capability, sophisticated applications such as Web site personalization will not be possible. Furthermore, as e-business models continue to evolve at light speed, the high-performance data solution can adapt to changing business rules and Web applications that turn over every 24 months.

The ability to capture and use data for competitive advantage will be a key determinant of e-business success. Enterprises that effectively manage their data will facilitate strong relationships with customers and trading partners, while enterprises that let data overwhelm them will alienate customers and lose business opportunities. Scalable, high-performance data solutions are the only way to derive full value from the rich supply of data created by e-business.


1 GartnerGroup. Electronic Commerce Conference Proceedings. February 29 ­ March 2, 2000.

2 Ibid. GartnerGroup.

3 Evans, Rich. "Operating the Move to Data Ware-housing and OLAP." META Group. January 17, 2000.

4 Ibid. GartnerGroup.

5 Ibid. GartnerGroup.

6 Ibid. GartnerGroup.

7 Nicolett, M., Andren, E., Gilbert M. "Challenges of Aggregating and Managing Catalog Content." GartnerGroup. April 12, 2000.

8 Schmitt, Eric. "Measuring Web Success." Forrester Research. November 1999.

9 Ibid. Evans.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access