My 19-month-old son faces several major dilemmas on a daily basis. One of the most peculiar and frustrating to him (and his parents) is the paradox of cleanliness. My son cries pretty loud when he falls down and gets mud on his hands; however, he is no fan of the bathtub. Currently, baths are taken standing up and screaming in order to minimize contact with water and to ensure that mom or dad will be as efficient as possible.This whole ordeal reminds me of the process that many of our clients are going through in order to clean their customer information. All of our clients want integrated, cleansed and deduped information. They would like to get the mud off of their hands. However, most realize that this is a large undertaking and the water can be really hot. Outsourcing this capability makes quite a bit of sense and takes a huge amount of responsibility off the company's shoulders. However, outsourcing is very expensive in the long run. What are the tradeoffs for both initial development and ongoing maintenance and support?

Initial Development

The initial entry into customer information cleansing is a large endeavor. A few basic tasks are required regardless of whether you buy technology or outsource the process:

  • Design a customer hierarchy.
  • Identify the matching rules to identify when two customer records can be considered the same customer.
  • Identify what will be considered the best record at each level of the hierarchy. For instance, a telecommunications company may have several accounts for a customer, one for each product - long distance, wireless and broadband. They will want to take the best information from each account to create one "individual" record. Maybe one account has an email address but the rest do not. They will want their best record to have an email address.

Identifying and designing these rules can be time-consuming, and the decisions are not trivial. In an outsourcing scenario, the vendor typically has many of the best practices built into its processes and can easily accommodate modifications to those standard processes.
In contrast, leveraging off-the-shelf technology and customizing it for your specific needs will be starting from scratch compared to using an outsourcer. Those rules identified during requirements need to be configured and coded into a data cleansing tool.

The bottom line is that most outsourcing vendors will be able to streamline initial development and have you up and running in a production environment long before an in-house solution will be ready.

Ongoing Maintenance

Anybody who has worked in the customer information cleansing world for a while knows how iterative both development and support can be. No organization should be so bold to say that they will get all of the rules right on the first try. Rules will evolve over time, and the technology implementation will need to roll with those changes.

Likewise, organizations need a plan as the rules change. Households will break apart, and consumers who were not matched into a household will be merged after the fact. How does the customer data warehouse resynchronize all of the transactions, orders, returns, subscriptions and promotion history to these new households or consumers?

Over time, many organizations find that an in-house solution to customer information cleansing allows them to be more nimble to integrate new data sources, change rules and resynchronize the database. The scare of excessive professional services from an outsourcer to continue to customize cleansing rules is also front of mind for many organizations.

Making the Choice

In the end, organizations will be successful with either an outsourcing relationship or in-house implementation. The main questions organizations need to ask themselves are:

  • How fast does the implementation need to be built?
  • Does the organization have the manpower and expertise to get the solution "right enough" on the first try?
  • How flexible will the system need to be post-implementation?
  • What are the current and ongoing costs associated with development and changes? What are the costs in emergency situations?

Nobody likes to take a bath - neither a toddler inconvenienced by soap, nor an organization inconvenienced by disparate systems. However, for everyone's health, you need to be clean.
Larry Goldman is president of AmberLeaf, a customer intelligence consultancy. He can be reached at

This article originally appeared in DM Review.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access