Continue in 2 seconds

Where’s the Info?

Published
  • July 06 2000, 1:00am EDT

Touchpoints. Point of customer contact. Customer interaction. Personalization. One-to-one marketing. 360-degree view of the customer. Holistic view of interactions. One-stop shopping for customer information. Multichannel coordination.

Without a doubt, organizations are starting to migrate into real-time interaction with customers. With the right customer information at the right time, we can produce the right offer, provide the right treatment and/or the right service. This is the new nirvana, the new goal... and just as we got good at campaign management!

The idea of having the right information at the right time is appealing to most marketers. New technologies from the Web to customer service are enabling front-office workers to make intelligent decisions around customer interactions. The key to enabling this functionality is the information itself. Whether it is automated personalization on the Web or simply making sure that a customer's history is easy for a salesperson or customer service representative to view, the information now drives the process.

Organizations are struggling with the data architecture to make this concept a reality. The corporate information factory dictates that an operational data store (ODS) is the answer. The ODS stores real-time or near real-time, mostly current, integrated information sourced from a myriad of transactional systems. Though tactical reporting may be performed on the ODS, serious strategic reporting or CRM-related activities such as campaign management would be performed in the data warehouse where more history is stored. However, in the context we are talking about, the ODS would be very useful as an integrated customer database with which different touchpoints which could be leveraged for real-time personalized service or offers. Consequently, the Web site, customer service or direct sales would all be running off the same integrated, customer database with the activity data from each touchpoint available to the others.

The issue many organizations may ask is, "Can I use one of my current transactional systems as the ODS, or do I have to build a brand new database?"

This is an excellent question and one that deserves a good deal of thought. Why build a new database when there may be an existing database that is 20 percent completed, 50 percent completed or even 75 percent completed? It is difficult to justify the development, hardware and software for a brand new database. The investment could very well be huge, and if it is supporting real-time queries from transactional systems, it will also require industrial-strength hardware. (Of course, you will also need to replicate the information for the data warehouse.) This is a very tricky situation.

Figure 1 helps us identify the differences between ODSs, operational systems and data warehouses.

Focus Business operation Business management Business intelligence
Data Content Current Current Current, historic, summarized, derived
Data Organization Process based Process based Data based
Data Scope Function, subfunction Cross functional Cross functional
Nature of Data Highly volatile volatile Static, historical snapshots
Data Update Field by field Record by record Batch
Access Individual rows Individual rows/small subsets Small and large sets
Usage Highly structured Structured, some analysis Unstructured and unpredictable
Response Subsecond to seconds Seconds to minutes Minutes to hours
Philosophy Run the business Integration to support tactical activities Integration to support strategic activities
Priorities Availability and performance Availability and performance Flexibility and access

Figure 1: The Differences between ODSs, Operational Systems and Data Warehouses

This chart outlines some of the distinctions between transactional systems and the ODS; however, it also highlights the similarities between the two systems. The issue is confused by the fact that most companies have a system that houses quite a bit of their customer profile information (though maybe not all of the transactions). For example, many telecommunications companies have a very strong billing system that is required to store not only customer profile information but many of the very important transactions that would drive segmentation and customer valuation. A Web-based company or Web department of a brick-and-mortar company may have a main database that runs the Web site that holds customer information and purchases. If these databases were the ODS, it would save them time and money.

Some companies have fully embraced operational customer relationship management suites. Firms that have invested in large-scale implementations of Clarify or Siebel may house quite a bit of information about customers and interactions with these customers under one database where these same suites control many of the touchpoints.

Architecturally speaking, what do we do? Do we use the transaction system that is "almost there," or do we build an ODS? As usual, it will depend on the specific company, but I have provided a few rules of thumb and questions you should ask yourself.

Scalability

If the transactional system is going to be used as the main customer database, it will need to handle the volume if other transaction systems start querying real time. Remember, one of the main reasons for offloading data warehouses is the stress it puts on mission critical transaction systems. You should consider the following:

  • Will the hardware need to be upgraded?
  • Will the software need to be upgraded? How can we be sure?

In this case, where the transactional system is used as the main customer database, it may make sense to invest in some performance testing tools to simulate the transaction volume. Though an ODS is mostly loaded with near real-time data, there may also be some batch interfaces. You need to think about whether the transaction system can handle more processes in its batch window or if it will disrupt the schedule.
Integration

Consider:

  • How much of the overall customer information does the operational system really have?
  • Does it include all of the profile information?
  • Does it include all of the transactions?
  • Does it include all of the customer interactions?
  • Does it include customer-scoring algorithms?
  • How clean is the information?
  • How hard will it be to integrate the information?

Integration is the main concern during this decision process. Basically, if the database only contains 25 percent of the customer information is it worth bringing the other 75 percent of the information over to this database? In this case, it would probably make more sense to create an ODS, because you would be moving, transforming and cleaning most of the data anyway; and it gives you the opportunity to make minor transformations and validations on 100 percent of the information. As a rule, the transaction systems should probably only be considered to be the ODS if it has more than 50 percent of the information.
The level of difficulty of integration is the other point to consider. This is also where gut instinct is important. The main advantage of a data warehouse or an ODS is that it is integrated. We always used to say that the data warehouse produces the impossible report because of integration. It just would not be feasible to create these types of reports without the data automatically integrated.

Keep in mind that by using the transactional system as the ODS, you are forced into a certain design, and you need to make the rest of the information conform to the current database design. Just because the word "operational" is in ODS, it does not mean the design will be strictly normalized. Since the ODS will still mainly be queried, you may take advantage of the situation and optimize the database differently. Without changing the current structures in the transactional system, will you be able to attain the level of integration you need to provide personalization or multichannel coordination?

Additional considerations to evaluate include whether or not you are able to modify the operational structures without breaking the transaction system and whether or not you can you add data to the transaction systems table without breaking the system. The transaction system runs off of a set of business rules. Adding data from other systems into the current structures may not abide by these business rules and cause the records either to be rejected or possibly "break " the application. Bending these rules may not be the right answer because it may hurt the integrity of the transaction system. Remember, rules are there for a reason.

Obviously, this point is probably the most problematic. An ODS is nothing without integration and creating a separate database gives the ODS designers complete flexibility. Integration and data quality will not suffer due to baggage of a legacy system.

Organizational

In the scenario where we create a separate database, the main problem is, from a design perspective, the system is torn in two directions. It must continue to be the OLTP for its function (billing, customer service, etc.), and it must act as an ODS. From an organizational standpoint, the same problem occurs. Common questions to consider include:

  • Who is technically in charge of the database – the ODS team or the OLTP team?
  • Does the OLTP technical team have the resources for the new functionality? Do they have the skill sets?
  • How are enhancements to the system prioritized across needs for the OLTP and the ODS?
  • Who wins standoffs?
  • What is the escalation path to have the problem resolved?
  • Who owns the ODS project plan and initiatives? Is this a separate team, or is it the OLTP team?

Openness

This point is very simple. The ODS needs to communicate with a variety of platforms in real time. Consider:

  • Is the transaction system able to communicate (in real time) with a variety of platforms?
  • Do enterprise application integration tools such as Active or Vitria work on the OLTP platform?

Touchpoints

Consider:

  • How many touchpoints does the transaction system own?

This question is of particular interest in the cases of implementations of front-office suites such as Siebel or Clarify. If the suite owns the majority of the touchpoints, already communicates across all of the channels and runs off of a centralized database, then it is very close to being stamped as an ODS. To finish working through the process, we would consult the previous points. However, in this case, it should be considered the ODS if many of the touchpoints are owned by the same system.
As a follow-up, if an organization chooses to use a front-office suite as the ODS, it must make sure that the rest of the corporate information factory architecture is not jeopardized. Many of the suites have out-of-the-box packaged data marts; however, moving from an ODS to a data mart is not advisable if strategic reporting needs are the goal.

Always Remember the End Goal

Access to common customer information by all touchpoints and all interactions from all touchpoints must be shared with each other. Leveraging existing infrastructure is extremely important, and the pressure to get an ODS to market may be unbearable. But, before you load hundreds of gigabytes into your OLTP systems be sure to take some time to think about it.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access