CATEGORY: Data Warehouse Design, Management & Performance
REVIEWER: Jay Graves, president of SmartDM.
BACKGROUND: SmartDM is a Nashville-based direct marketing firm serving clients in vertical markets that include financial services, nonprofit organizations, hospitality, healthcare, retail and entertainment. SmartDM offers three main channels of direct marketing services including direct mail, e-mail and data management. These channels provide clients with a fully integrated services platform that takes direct mail and e-mail marketing campaigns from concept to execution. With DirectTR@K, our company’s ASP-based data management and CRM application, organizations gain data warehousing capabilities including the ability to create an up-to-date view of all relevant marketing information, perform queries, pull lists and research trends.
PLATFORMS: Data Junction is running on Windows 2000.
PROBLEM SOLVED: Our clients include sports teams in the National Hockey League, Major League Baseball and the National Collegiate Athletic Association. SmartDM gives these organizations consolidated views of their customer data through our proprietary data warehousing and marketing application, DirectTR@K. To implement DirectTR@K, customer data, including feeds from ticketing systems, is fed into the application. This data comes from many different sources and is delivered in a wide variety of formats from ASCII delimited to XML feeds, and proprietary formats with unconventional structures. In order to bring this data into our SQL Server-based application, it must first be cleansed and normalized. We also need to load and refresh this data quickly to give our clients a current view of their customer data. We have addressed these two challenges effectively by implementing Data Junction Integration Studio as our ETL tool. Data Junction’s Map Designer gives us the ability to take data in virtually any format and transform it into SQL Server. By deploying the Integration Engine, we have automated daily scheduled ETL processes that enable us to refresh customer information in clients’ warehouses.
PRODUCT FUNCTIONALITY: Having so many different kinds of source data, we rely on the extensive functionality in Data Junction’s Map Designer to transform and cleanse sizable data feeds. With the Expression Builder, we are able to apply business rules to manipulate data during the transformation process. There is extensive data hygiene involved with managing multiple data feeds into DirectTR@K. This includes merge/purge processes, CASS certification, duplication elimination and standardization of key data fields. We use Process Designer to build these steps, including transformation maps, into one executable process and the Integration Engine to automate these processes. We are then able to schedule these processes to run during off-peak hours. With ETL processes set up through Process Designer we gain data quality that is essential for optimum use of our application. We also realize significant time-savings in implementing the application and in refreshing information.
STRENGTHS: We consider the main strength of Data Junction to be the Process Designer and its ability to work with the Integration Engine. With the functionality offered in these tools, we have been able to set up a routine through which our database checks every five minutes for new jobs and then responds with a specific automated ETL process. This provides our clients access to customer information from disparate sources quickly and in a standardized format.
WEAKNESSES: The user interface could be more intuitive when working with complex processes. For example, if we create a variable in the process and need to use that dynamic connection for every transformation or export routing, it is necessary to recreate and initialize that process level variable inside each transformation. The next product release, however, promises the capability to choose, via an import option, which variable should be imported as opposed to recreating those variables across every transformation.
SELECTION CRITERIA: We chose Data Junction as our sole ETL solution because it offered all the functionality we needed at the price point we were considering.
DELIVERABLES: Data Junction currently enables us to support at least 15 continuous feeds of data that differ extensively in format and/or structure. These data feeds vary in size from 10,000 rows to 2 million rows at a time. With Data Junction we are able to bring information from any source into our warehousing application and give our clients swift access to that information.
VENDOR SUPPORT: Vendor support has been great. The best part of working with Data Junction’s Support Center is that we can get a human being on the phone. Their technicians are knowledgeable and respond quickly.
DOCUMENTATION: Data Junction’s documentation is good, especially the help files. These offer clear explanations of how to work with the tools as well as helpful examples of map and process designs.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access