CATEGORY: Data Warehouse Design, Administration & Management

REVIEWER: John Markham, product development director at Identex.

BACKGROUND: How do companies such as AOL, British Airways, Yellow Pages and Citroen get closer to their customers? They turn to Identex, customer management and marketing automation specialists. Identex offers companies a wide range of services that includes everything from sophisticated marketing campaign automation and database management to state-of-the-art information quality management. Utilizing best-in-class marketing software, Identex is able to leverage its clients' data into a true business resource for closer, more profitable customer relationships.

PLATFORMS: Our platform consists of an NT-based system with an eight-way CPU running EMC storage network client. The database is primarily SQL Server 2000.

PROBLEM SOLVED: We store our clients' data, including name and address records as well as database information such as a customer's purchasing history with a company in a single, secure location. These records are put through Identex's own software to cleanse, verify and update addresses. After that is complete, we do duplication processing on the data. With more than 100 million records to sort and manipulate each week (approximately six to 10 gigabytes), this can be a very time-consuming process. To cleanse and sort the large volume of records, MicroFocus COBOL was used. Because the data was originally held on a mainframe, this was a natural choice. However, once it was migrated to an NT-based system, it was soon realized that this was impractical for two reasons. Not only was the process taking too long, but the tools being used were unable to process individual records once they exceeded a certain length. A lot of sorting and data manipulation occurs during these processes, which is why SyncSort was chosen. SyncSort is used to sort files into the necessary sequence for processing after it has been through the data cleansing routines prior to merge/purge. This high-performance utility provides Identex with the solution needed for these problems.

PRODUCT FUNCTIONALITY: We are experiencing considerable time savings using SyncSort. It easily overcame the obstacle we had with MicroFocus COBOL in which we couldn't sort and further process beyond a certain record size. These limitations are gone with SyncSort. Jobs that would take hours to sort in the past are now running in what seems like a few minutes.

STRENGTHS: Overall the performance has exceeded our expectations.

WEAKNESSES: Previous experience with SyncSort on the mainframe raised concerns that everyone at Identex would be required to learn the technical aspects of the program. However, Visual SyncSort includes a GUI to create, maintain and run SyncSort applications. After testing the program, we decided to roll it out on the desktop. Just by using the GUI, people were immediately able to set up batch jobs without having to know any specific coding. It is much easier and faster, but still achieves the same high-performance results as SyncSort for Windows/NT. For example, the production controllers now do all their data manipulation simply by using the GUI.

SELECTION CRITERIA: SyncSort for Windows/NT, a high-performance data sort and manipulation product from Syncsort Limited, a subsidiary of Syncsort Incorporated, was recommended due to the considerable amount of time saved in data sorting/manipulation.

DELIVERABLES: After initially implementing SyncSort for file sorting, we found additional uses for the software. Another product was being used to conduct file manipulation. A file would be read in, the data manipulated and a file written into a specified format. The manipulation was done by writing routines in a programming language. For example, a program would be written for matching a customer file before it is processed in order to review the results. SyncSort is now used instead. It saves us from having to code error-checking routines to catch potential mishaps with matching two files. Additionally, the speed at which SyncSort compresses the data is phenomenal.

VENDOR SUPPORT: When required, SyncSort's technical support team proved efficient, helpful and effective in helping with a couple of issues we had after we licensed the software.

DOCUMENTATION: The documentation was very good and met all our needs.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access