More vendors are now providing data profiling solutions. Some products provide relatively general capabilities while others are embedded within larger application suites with specific data targets. Apparently, a large part of the data profiling capability is now routinely (and, to some extent, trivially) integrated into other applications. Additionally, most major data standardization and matching tool vendors have deployed a data profiling solution through partnership, development or acquisition. Data profiling is moving into the mainstream and becoming a fixture in numerous data management processes, ranging from system migrations, data warehousing and data quality to operational system improvement.

Data profiling is used to explore anomalies within a collection of data sets and to expose potential problems inherent in the data but not explicitly stated. Profiling is also used to review compliance of a data set with its documented meta data, as well as conformance with a data model. Most of the existing vendor products provide some kind of business rule definition, along with auditing and reporting functionality. However, without getting into great detail, data profiling technology consists of three major capabilities. The first, frequently referred to as "column profiling," provides statistics and analysis about the values assigned to the attributes within each column in a table. The second, which has been referred to as "redundancy profiling" or "cross-table profiling," reviews and explores relationships between columns in different tables, with the expectation of discovering foreign key relationships as well as violations of referential integrity constraints. The third, referred to as "dependency profiling" looks for functional dependency relationships that exist across columns within a single table.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access