BACKGROUND: The Bureau of Land Management (BLM) manages one-eighth of the land in the United States and maintains approximately 1 billion land and mineral records. In 1998, we had just eight months to rehost our massive land and mineral databases to a Y2K-compliant system. This challenge was complicated by data quality problems including inconsistent data formats used by 12 different states.
PLATFORMS: The legacy data was in flat COBOL files on an old Honeywell system. There were 35 tables used by four different applications. The total databases included more than 150 million rows and 20 gigabytes of data, and the largest tables had up to 27 million rows. There was very little documentation available. We planned to move the data to a transaction-processing database on a RISC6000 and a reporting database on a Sun E10K system.
PROBLEM SOLVED: We were all keenly aware that a recent BLM automation project had failed because its target data model didn't fit the actual source data. We knew that manual data analysis could only look at about 0.1 percent of the data, providing a very limited knowledge of its content and structure. Clearly, we needed an automated tool that would give us a total understanding of our data it was the only way we could be certain the data model we were building in the target system would accommodate the legacy data. That's why we turned to Migration Architect, the data profiling and mapping solution from Evoke Software. Migration Architect gave us a window into the data and helped us complete our Y2K project right on schedule.
PRODUCT FUNCTIONALITY: Migration Architect uses advanced inference and pattern matching techniques to reveal the content, structure, quality and integrity of legacy data. Migration Architect examines each data element and reports detailed characteristics for each column, such as data type and size, range of values and frequency, and distribution of data values. The product also identifies dependency relationships between data elements, and it finds redundancies between different tables. Migration Architect revealed inconsistent data formats and other anomalies, and it helped us produce programs that accurately moved data from the legacy system to our new Y2K- compliant system.
STRENGTHS: Migration Architect provided excellent data profiling reports that gave us a thorough understanding of our source data. When you have mountains of data, you can't possibly do the analysis manually. Evoke gave us a tool that enabled us to thoroughly analyze all of our data, and this led to project success. We have been so pleased with Migration Architect on the Y2K effort that we purchased additional seats. We are now using it on the National Integrated Land System, a project to automate cadastral land record information.
WEAKNESSES: It would be great if Migration Architect had a command-line interface so it could be automatically launched as a part of a scheduled data quality management program.
SELECTION CRITERIA: We needed a tool that would give us a complete and accurate profile of our data. We also needed an efficient tool, since we were working with a small team (12 people) and a short time frame. Migration Architect was the only solution we found that would address these needs.
DELIVERABLES: Migration Architect provides a wealth of information in the form of online screens, drill downs, reports and spreadsheets. The results can be loaded in the Evoke Repository, and you can use third-party reporting and analysis tools to further investigate the data. The reports often revealed unexpected values and inconsistencies between the data entered at different times and by different administrative regions. The reports enabled us to ensure that the target database would be able to handle the various types of data it would receive.
VENDOR SUPPORT: Evoke has been very responsive to our needs, and they have added new features at our request.
DOCUMENTATION: The Migration Architect documentation was very helpful during our on-site training program and continues to be a useful resource.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access