REVIEWER: Steve Hromyko, ETL manager for LifeMasters Supported SelfCare, Inc.
BACKGROUND: Founded in 1994, LifeMasters Supported SelfCare, Inc. is a leading provider of health promotion and disease management programs and services that create health partnerships among individuals, their physicians and payors.
HARDWARE PLATFORMS: Intel-based servers running Microsoft Windows 2003 Server and MS SQL 2000.
PROBLEM SOLVED: LifeMasters receives and processes large amounts of data from a broad spectrum of sources. With growth in business came the realization that data volumes would ultimately outpace their manageability. To prevent this, we needed a set of tools that could not only digest and transform a variety of file types (ASCII, COBOL, EDI), but that also provided added functionality to enable full lights out automation of our loading process. Pervasive Data Integrator gave us this capability. We now pull files from a secure FTP server, decrypt, stage, transform and load them to our SQL databases without the need for human manipulation. This significantly reduces the time required to process data, and the data itself is inherently more secure. Analysts can spend more time turning this data into usable information.
PRODUCT FUNCTIONALITY: We handle more than 120 different file formats, many of which require unique transformation logic. Using Pervasive Process Designer, we are able to build a single master process into which any file from any source can be fed. Upon receipt, the file is married (programmatically) to its associated transformation (mapping logic) and loaded. This relationship, along with additional information needed to manipulate the file, is stored in metatables and gathered during runtime using RIFL, Pervasives scripting language. To maintain simplicity, we made an assumption that a data file could be associated with one and only one transformation. When processing a file, no matter how complex the layout, we will use a single mapping step to transform and move the data into a standard set of tables. This can create challenges when working with files containing complex looping structures, but Pervasives powerful Map Designer makes this possible.
STRENGTHS: With Pervasive we are able to use our existing processes rather than rebuild from the ground up. The Process Designer provides components to access existing technologies (SQL procedures, DTS routines, custom applications) so we can leverage the skill set of our employees and reuse what is already built and works well. This enables us to automate now and optimize over time. Out of the box, the Map Designer provides considerable built-in functionality to implement transformations quickly, without the need for development resources.
WEAKNESSES: Now that our users are proficient, we are continually finding better, more creative ways to use the tools. To that end, having access to samples, use cases and insight into how other customers are using the tools would have enabled us to get to the point we are at even faster.
SELECTION CRITERIA: Our main focus was on data connectivity, data quality and automation. Our company works with a variety of file types, but our focus is toward industry standards. Pervasive provides our analysts with a means of reading health service-related file formats, such as HIPAA, HL7 and NCPDP, that did not previously exist. We also wanted the ability to assess the quality of data received from our clients prior to loading. Pervasive Data Profiler lets us generate metrics quickly and easily so that files can be analyzed and, if need be, recycled back to the customer before the loading process begins. Finally, we were leery of replacing a manual process with one partially automated that over time would become just as labor-intensive. Pervasives comprehensive set of tools provides a complete end-to-end solution to facilitate full automation.
DELIVERABLES: Using Pervasive Data Integrator, we will be able to reduce the time it takes to process files from days to hours. Further, as the automation becomes more sophisticated, the time required by operations personnel to monitor this process will approach zero.
VENDOR SUPPORT: We enjoy a great relationship with the Pervasive support and services teams. Not only are they knowledgeable and proactive when working with us to resolve issues, but they listen to us as a company and are genuinely interested in how we use their software for our business. A number of enhancements have made it into their tools based on our feedback and ideas.
DOCUMENTATION: The documentation is satisfactory, but with such a flexible tool there are areas where having additional examples would extremely helpful.
Pervasive Data Integrator 9.0
Pervasive Software Inc.
12365-B Riata Trace Parkway
Austin, TX 78727
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access