Once a local and customer-centered activity, the art of marketing has developed explosively over the last century. The first breakthrough came with the advent of mass communication technologies, such as radio and television, which expanded the audience reached by marketers to millions of people located throughout vast geographic areas. To reach such a large audience necessitated the use of a universal marketing message. Marketing became product oriented for a while. The second revolution in marketing was ignited by the rapid progress of computers and databases. This endowed corporations with memory: data about customers and transactions was now stored for future use. The idea was to utilize historic data to make marketing customer oriented again – and thus more efficient.

However, it was soon realized that the amount of data to be analyzed in order to make better marketing decisions requires completely new, automated data modeling techniques. The concept of data mining – automated search for new, useful and actionable knowledge hidden in data – emerged in the nineties. Data mining is still in its infancy – but is very promising. Incorporating technologies for automated learning from historic data into complete decision support systems for direct marketing has the potential to close the loop. The world will return to customized direct marketing to individual prospects, but this time delivering an individually focused message to each of millions of prospects at once.

New direct marketing techniques provide means for custom tailoring the marketing message and channeling it to precisely those prospects likely to respond to the promoted product or service. Modern marketers rely on decision support technologies to facilitate an accurate match between the marketed products or services and potential consumers, thus improving the efficiency of a marketing campaign and saving the company significant funds – an important competitive advantage.

Direct marketing tasks include market segmentation, response modeling, cross-sell prediction, customer valuation and market basket analysis. Building successful solutions for these tasks requires applying advanced data mining and machine learning techniques to find relationships and patterns in historical data and using this knowledge to predict each prospect's reaction to future situations.

Since a whole new generation of marketing managers is embracing the ideas of one-to-one marketing and mass customization, one would expect to see an abundance of reliable decision support applications for direct marketing. However, this is not the case. What technological problems have been slowing down a universal acceptance of the new custom marketing techniques, and how can these problems be solved?

To find an answer, let us consider a typical direct marketing workflow chart (see Figure 1). The shown workflow underscores that direct marketing is a multi-step recurrent process, using customer feedback and gaining precision with each round of customer communication, data collection and analysis – not a one-time endeavor. The direct marketing chain involves the steps of data acquisition, storage, modeling and scoring, followed by campaign management and customer response measurement. These steps have been linked and computerized to a large degree, resulting in the first integrated direct marketing software solutions. The bottleneck of this process remains in the step of data modeling and knowledge discovery: significant human intervention is required at this step.

The direct marketing chain starts with capturing data about prospective customers. The most common sources of such information are e- commerce Web sites, point-of-sale scanners in stores and customer loyalty programs, which allow tracing sequential transactions of individual customers. Also, data can be purchased from external database vendors as a supplementary or even a primary data source. In order to analyze captured data, it is necessary to move transactional data to a data warehouse to facilitate data consistency and perform various aggregations. After that, OLAP tools are used to create views of data suitable for further analytical analysis. This part of the direct marketing workflow has been largely automated and integrated with campaign management tools.

Let us assume for now that we know how to carry out the step of data mining and find hidden patterns in data, as well as explanations of these patterns and accurate and significant models predicting outcomes of future situations. The next step is to use the developed model to score the bulk of customer records in the database to pinpoint the best prospects, segment the market and identify promising cross-sell opportunities. These predictions serve as a basis for the future marketing strategy, allowing us to operate with the most efficiency and wisdom. Finally, we communicate with selected customers and measure the response, recording it in the data warehouse to make our marketing solution smarter with each round of direct marketing.

The Bottleneck

Data analysis currently represents the most difficult and nonintegrated step of the business intelligence chain. An intervention of a human analyst is required on several occasions during the data modeling stage. First, a sample of historic data for analysis and machine learning has to be imported from the existing storage architecture to a selected data mining application. Then the best strategy for data analysis has to be selected. Finally, the developed model needs to be exported to the direct marketing application in order to score the bulk of new data and identify the best business opportunities. Data mining remains the bottleneck of direct marketing solutions because reliable machine learning algorithms aimed at distilling models and patterns from data involve complex mathematics and have just emerged from academic circles to the business world. The corresponding software products are immature, and they possess no means for simple integration with other pieces of the decision support chain.

There are quite a few standalone analytical tools available on the market. However, data mining applications have to process data of a diverse nature drawn from different storage architectures, use multiple data-specific exploration algorithms and present results in a variety of forms suiting the needs of individual customers. Consequently, data mining applications come to the market today either overloaded with functionality which is never used in its entirety by any single user or relatively simple and universal – but very hard to customize for a specific problem.

In fact, there is a combination of issues that complicate successful incorporation of the data mining link in direct marketing chain. Often, data mining systems are too complex to operate, lack algorithms suitable for tackling specific direct marketing tasks, cannot be simply integrated into a complete decision support chain or are quite expensive (hovering in the range of one hundred thousand dollars or more). However, there is a strategy that addresses all these problems at once.

The Cure: COM Data Mining

This strategy suggests that data mining applications should follow the specialization and unification example already set by hardware manufacturers. As computers are readily built from separate hardware components, new software applications should be easily built from standardized software components. This new software development paradigm, called Component Object Model (COM), was specified by Microsoft in the end of 1995 and is also known as ActiveX technology.

COM defines a standard mechanism that allows different software components to provide their services to each other. This mechanism works the same way in all possible situations, independent of the nature of the involved components. The technology provides a foundation for interface-oriented programming: if you know how to communicate with a component, you can use it.

Recently, a few manufacturers of data mining tools rebuilt existing data mining algorithms on the basis of ActiveX (or COM) components, which can be readily incorporated in any existing vertical decision support system (DSS), boosting the performance of the system as a whole. This makes the data mining step an integral part of the direct marketing business intelligence chain as depicted in Figure 2.

The functionality of an integrated, COM-based application can be illustrated by the scheme displayed in Figure 3. The main component is, in this case, an integrated direct marketing and campaign management solution which relies on the services provided by the data mining component. The user takes full advantage of the powerful data mining algorithms provided by the knowledge server while using a familiar interface provided by the integrated direct marketing solution for communicating with these algorithms, visualizing the data and applying the results of a performed data exploration.

The requests for data analysis are processed through joint work of different components. The client component furnishes a seamless interface with other parts of the integrated DSS and addresses the data access component responsible for the import of data for the analysis. Then the client component calls the methods of the server component implementing the required exploration engines and provides a pointer to the tuned data access component. The server performs the data exploration. The obtained intermediate and final data exploration results are passed to the client component for visualization and further manipulation.

The COM architecture allows the use of all the power of the analytical algorithms built into the data mining component and all the data manipulation and presentation functionality provided by the complete decision support application. This is a crucial step forward since all that the users really want is the ability to access the new data mining capabilities from their favorite application.

Do-It-Yourself Integration

From a developer's perspective, ActiveX components start to look even more appealing. Now developers and integrators can:

  • Create powerful new applications quickly and effortlessly.
  • Incorporate third-party components in the created applications.
  • Shop for the best components from different vendors.
  • Extend the functionality of existing applications by simply adding new components.
  • Combine components written in different environments and languages in a single solution.
  • Carry out the integration with the simplest and most common tools (such as Visual Basic for Applications).

On the other hand, manufacturers of data mining components will never again have to drive themselves crazy trying to build a complete vertical decision support application on their own; they were not very good at that anyway. Now, with a sigh of relief, they can concentrate on creating well-documented, COM-based data mining components, leaving their incorporation into vertical solutions to the professionals with expertise in that area. Data mining component developers should get ready to welcome their former competitors as their new customers.

The Beauty of COMponents

Summarizing, the utilization of COM-based data mining algorithms at the data modeling step of the integrated direct marketing chain serves the needs of all interested parties. Users obtain a flexible architecture of individually purchased, powerful data mining algorithms, integrated seamlessly into their DSS. Developers and integrators of vertical applications can hire cheaper VBA programmers to replace their C++ gurus and concentrate on perfecting the user interface while outsourcing for efficient machine-learning components. Developers of data mining algorithms can sell their components in a broader market, striking new strategic partnerships with the developers of the best direct marketing solutions. Business intelligence solutions become flexible, easily upgradeable and cost-efficient. A combination of these features boosts the customized value of applications built with the help of COM technology, at the same time expanding the market for the created components to the broadest possible level.

Finally, there is another very important group of people that benefit from the new technology – the targeted audience. New integrated direct marketing solutions will provide a better segmentation and targeting of prospects and offers in a marketing campaign. We all should expect to receive fewer pieces of junk mail while getting promotions for things we really need. That's how we will all know the new technology is indeed working!

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access