By defining intersections in database extracts and legacy files, CoSort users can simultaneously discover, transform, and report on related data. And by performing join and lookup functions on flat files, CoSort users can: 1) relieve the DBMS of query overhead; and, 2) incorporate mainframe/index file, spreadsheet and other data into the process.
Multifile Joins
Joining large tables to satisfy queries taxes DBMS performance. There has also been no efficient way to compare large files and identify field changes (inserts, updates, deletes) over time. "In addition to offloading DBMSs, multifile joins offload data integration tools, by merging data before it hits the tool," said Philip Russom, senior manager at The Data Warehousing Institute. "At the high end, this is useful with the distributed architectures that many users apply to scaling up their data integration solutions. At the other extreme, multi-file joins may eliminate the need for a data integration tool."
Multidimensional File Lookups
Data cleansing, multitable joins and complex computations that produce discrete solutions are resource-intensive operations. Where a simple lookup can replace a runtime computation (e.g., mathematic expression or pseudonymization), the performance gain is significant because retrieving a value in memory is faster than computing that value. To achieve these fast retrievals, CoSort users specify lookups against set files. By referencing multicolumn files, users get faster answers to discrete questions like the right ZIP code for a city in a state lookup. Russom added that "when multicolumn files are sources for a data warehouse, multidimensional file lookups can generate cubes and other multidimensional structures for the warehouse and analysis tools."
This piece is brought to you by the Information Management editorial staff.
Be the first to comment on this post using the section below.