"Beyond the mountain lie more mountains" ­ Hindu proverb

During the 1980s, companies succeeded based on their ability to reduce layers of the corporation through restructuring. In the 1990s, companies that exploited their core operational and marketplace competencies achieved rapid growth. In the 2000s, companies that can engender the highest levels of customer satisfaction through the effective use of knowledge will be the winners.

Customers are now in charge. In many industries, there is no longer market space to place a profitable new operation. To achieve top-line growth, it is imperative that companies get the most out of existing customer relationships. Customer relationship management (CRM) is all about generating high levels of profitable customer satisfaction through the use of knowledge generated from CRM applications using corporate and external data.

Fortunately for many companies, the data warehouse provides a starting point and the data foundation for enabling CRM applications such as fulfillment, customer segmentation, targeted marketing, cross-selling, customer loyalty, profitability analysis, integrated billing, market basket analysis and other forms of combining customer touchpoints with external data to generate a valuable business asset.

However, many of today's data warehouses are not ready for the challenges of CRM, and steps must be taken to ensure readiness. This involves additional or expanded subject areas, external data and granular, high-volume accessible data. The biggest challenge in data warehousing today is supporting data warehouse expansion to provide a foundation for CRM. Traditional approaches, tools and processes may not work. Data warehouses that meet the needs of CRM are aptly named customer data repositories (CDR) due to the need for low-level, granular customer transaction data. This need has many implications for the data warehouses that need to support CRM.

Many have predicted the midpoint data warehouse size will be as much as 10TB by 2001. Many warehouses are already growing to these levels in order to support CRM. The CRM-ready data warehouse will be large and play by the rules of very large databases (VLDBs). Its dimensions will comprise customer, product, time, geography or site, promotion, prospect and a host of others. It will almost certainly have summary tables in order to support the majority of queries against it. Though most ad hoc and application queries will use summary data, the CRM-ready data warehouse must have detailed data in order to provide the foundation for CRM needs such as applying rankings to customers based on specificity of purchase at a product, day, store, checkout lane and perhaps even time-of-day level, to give an example from retail.

Eighty percent of the ROI from a CRM-ready data warehouse is generated from direct use of the detailed data or can be traced back to the warehousing of the detailed data. Capturing the data necessary for effective CRM means building a robust data warehouse containing three years or more of detail data with real-time access requirements. The customer precision that the CRM-ready data warehouse provides is equaled in importance by the speed of access to information that it provides. Effective use of scalable architectures, data partitioning and parallel architectures is paramount to success.

The data acquisition process can become exceedingly complex in this environment. Synchronous or daily feeds from hundreds or thousands of stores, clinics, distribution centers, banks, ATMs and other hubs of customer activity create an enormous number of failure points. Separating and dividing the process into manageable pieces that create meta data detailing each step's success or failure should source data not be delivered on time is a vastly underestimated step in creating the CRM-ready data warehouse. In such an environment, allowance for continuance of the load operation in the event some source data is not delivered on time is a must because available load windows are tight. Otherwise the data warehouse will always be behind. The rules governing the potential trade-offs between availability and recency of data should be brokered early on in such an environment.

The CRM-ready data warehouse will enable a closed-loop marketing approach by focusing on providing a corporate view of customers. Efficiencies of operation will be required; therefore, a central data architecture feeding data marts for specific purposes is highly recommended. Although multiple inferences can be generated from this approach, each one is generated from the same integrated and complete view of the customer. This complete view comprises a majority of customer touchpoints ­ purchases, promotions, payments, call center activity, incoming calls and Web site hits and activity. This has serious implications for e-commerce where a customer may provide numerous clicks and sit on numerous Web pages during an online session. External vendors can provide interpretation of such clickstreams through data mining.

This is only one area of the CRM-ready data warehouse where external partners are needed. Make room for external data! Reverse-appending customer demographics and other information into the data warehouse will be a necessity, and the volume of data available is enormous. Existing customers and prospects that fit a loose profile can enormously grow the customer dimension in terms of both columns and rows. An expanded attribute base will be required to do the deeper levels of customer segmentation necessary to establish customer profiling at a granular, actionable level. The CRM-ready data warehouse cannot rely on internal information alone, nor can it rely on static information. As customers and prospects encounter income and life-event changes, these changes need to be reflected in the data as well.

The lack of data quality is often cited as the number one data warehouse problem and will be even more troublesome in a CRM environment. Promotional decisions based on erroneous data can be very costly and often go undetected. Mailings are not possible to customers with missing address data. The aforementioned reverse appending of data will be hampered if data fields are missing or incorrect or if rows are duplicated. Consequently, it is not unusual to attach confidence levels to data and reports so that these confidence levels can be factored into data interpretation and decisions.

Do More With Less

Mass-marketing "spray-and-pray" approaches with maximum two percent returns are quickly giving way to smaller, targeted marketing approaches which will be executed more frequently and with higher, measurable returns utilizing multiple channels including e-mail and kiosk. Marketing departments everywhere are being asked to reduce budgets and return more top-line growth. This is forcing creativity and requiring iterative access to detailed and derived data. All this data must be sourced into the CRM-ready data warehouse at a level compatible with the other data in the data warehouse. Once that has been accomplished, the marketing possibilities today are unlike any before.

Although the granularity of data will be at the lowest levels in the data warehouse, applications will relate to various customer groupings such as households. Householding customers begins with quality detailed customer data. The grouping can be explicit through customer feedback, reliance on data broker groupings or, more likely, through derivation based on attributes such as last name, phone number and address. Groupings by econometric, demographic, psychographic, census and lifestyle factors are also valuable.

A characterization exercise yielding the exact makeup of the data is almost mandatory before reliance on the data can begin. This data quality review will allow for generation of criteria for logically segmenting the customer base. Percentiling, deciling, quintiling or quartiling, the usual quantiling options, may be most appropriate for each attribute depending on its value distributions.

Finally, and most importantly, processes for full utilization of the CRM-ready data warehouse by the business are required for overall success. These go well beyond stewarding the data. Corporate emphasis and incentives must reflect the value that CRM brings to the business. Turning customers into former customers through disincentives, once considered heresy, is now being adopted by companies as they focus on the profitability of individual customers rather than focusing strictly on the enabling goals such as numbers of customers. CRM-ready data warehouses can provide the empirical proof for such segmentation and business strategies.

Figure 1: CRM-Ready Data Warehouse Dimensions

Beginning next month, I will have a regular column in DM Review. Readying the data warehouse dimensions, as illustrated in Figure 1, and preparing architecturally and culturally for an active VLDB that supports your CRM goals will be explored in my columns. Other topics to be covered include the challenges that many organizations are now facing as they implement CRM programs and look to the data warehouse to provide the architectural foundation and the data.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access