Located in Minneapolis, Minnesota, Dean Hoff Consulting helps clients develop and implement analytic strategies. This includes building business intelligence applications utilizing software and other tools, then showing clients how the analytics can interact with their current processes. We specialize in the direct marketing industry, where we are able to demonstrate significant results such as increased response rates by uncovering critical information hidden in a company's countless gigabytes of data.

While working with a major direct marketing agency, we ran into a problem when developing predictive models. Part of the process required us to apply model scores to a large demographic file containing more than 100 million records. We developed the models and wanted to implement them, but in order to do this, we needed to be able to score the records. We checked if the company resources had the capability to accomplish this, but we were told we should outsource it because the size of the file was too large. After discussing possible solutions with a third party, we realized that the costs would have been too prohibitive. However, I knew there must be a way that we could do this internally.

After conducting an extensive amount of research, we decided to test DMExpress, the high-performance ETL (extract, transform and load) and data transformation product from Syncsort Incorporated. Because of the positive experience I had with SyncSort software in the past, I started to look into what Syncsort had done since that time. I found a lot of information about DMExpress on the company's Web site. The Web site also offers a free proof-of-concept for which Syncsort will actually build an application using DMExpress to prove the product will provide substantial performance improvements. After reviewing the information and discussing the project requirements with Syncsort, I determined that DMExpress offered the functionality and performance I needed for the project.

DMExpress allowed us to calculate a model score as well as score the large number of records in a short amount of time. We now take a historical marketing file and append the demographic information using DMExpress. Basically, we're matching the marketing file with a large demographic file using the join functionality. We then develop the model using the appended file, take the results of that model and score all of the demographic records we have. Once this is completed, the company has a breakdown of everyone in a geographical region who was contacted and whether or not they ordered a product. The information can then be used to improve the response rate of a direct marketing campaign by reaching out to consumers who are more likely to make a purchase.

One of the key benefits of DMExpress for us is its speed. Instead of taking days to run the aggregations, I'm able to complete the processing in a matter of minutes. Using the Advanced Data Management (ADM) component, DMExpress simplifies the creation, administration and execution of our aggregation jobs. It summarizes data much faster than other aggregation methods such as C programs, SQL statements or third-party multipurpose data warehouse packages. ADM high-performance aggregations utilize patented algorithms, state-of-the-art parallel processing technology and dynamic optimization to minimize the elapsed processing times. These time savings dramatically increase as data volumes grow, which is critical when we need to process the data but don't have much time because a client needs the results as soon as possible. Additionally, ADM is faster than other alternatives because it is powered by the DMExpress engine instead of database resources, which are not as efficient in processing data.

DMExpress also provided us with the functionality we needed to process data in other areas of the application. We initially integrated DMExpress into the application to just handle the scoring, but because of its join capability, we were able to use it to do the data append, too. With the aggregation component, we can actually do some profiling with the data as well so that we can determine any trends. Because of this enhanced functionality, we're able to use the product beyond its initial intent, which provides us with a significant return on our investment. 

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access