Organizations have built thousands of reports and “universes” over the years using their numerous BI tools. This is collectively a repository of latent knowledge that encapsulates business terms and their linkage to data. Wouldn’t it be great to mine this and put it to use?
BI tooling is being changed – regularly
How many times have we participated in the next "tool standardization" initiative? What was great a few years ago is now passé - organizations are often forced to change their tools due to business demands and vendor support. Very often, the "old reports" are not "convertible" to the new tool and building it again is the preferred route for most projects.
Of course, to complicate it further, the original report designers are not to be found and the current business user community would rather have the reports done "their way". Sounds familiar?
Metadata and "Catalog" efforts emerge due to Data Governance imperatives
Data Standards and the metadata to document these are being adopted within organizations at an increasing rate. These are driven by regulators or by disclosure requirements - where there is a great emphasis placed on consistency and quality.
These projects involve the creation of canonical data models - a process that is driven largely by data-domain technologists which do run the risk of becoming "ivory tower" projects. Efforts to ensure business participation and the availability of SME's are sometimes not successful as the business feels that this is "not our problem".
There is a better way - engage people differently and tap into the latent knowledge of BI
BI repositories are built up painstakingly over years to reflect the knowledge of the business - in the form of measure, dimensions, filters and calculations. Or perhaps a simpler approach was taken and all the business logic was in the report specification.
Regardless of the approach, institutionalized knowledge about the data and its meaning is very much embedded in these repositories and definitions. It would therefore, be very helpful to extract this latent knowledge, validate this with the business and use it as an important source of input to the data catalog and governance process.
How might one go about doing this?
We need to surface artifacts in the BI repositories and have a business-people engagementapproach that makes it easy for them to opine on data and definitions. A collaboration platform that has the ability to read commonly used BI repositories and underlying data structures, read the usage patterns in the data, guide the user community to the "experts" for curation and build out the canonical model based on actual users - would be the way to go.
Tools for metadata management and data governance are recognizing the need to absorb existing artifacts as well as to engage with the business community to "crowd source" their understanding. The new crop of tools excel variously in flexible metamodel schemas and ontologies, text mining and reading of repositories, machine learning methods of deriving metadata from the content as well as embedded collaboration capabilities.
The good news is that Metadata and Governance is a legitimate and recognized activity whose value is inextricably linked to the trustworthiness of the output. A collaborative process to build out these metadata standards based on knowledge that is often overlooked, has the potential to engage people, reduce the effort and make the quality to the work product very visible to management.
(About the author: S Ramakrishnan is an accomplished global financial services & IT business executive with demonstrated success in building and implementing market-leading data and analytics infrastructures & processes in the financial services Industry. He was most recently chief data officer at Citigroup).
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access