The silver lining related to bad data quality
Much of my working day focuses on helping clients tackle various aspects of data quality. Data quality is a broad term; I really mean data integrity across many dimensions such as quality, trust, ethics, privacy, availability, security, retention and so on. Collectively the umbrella term for all this work falls under is data (and analytics) governance; what we used to call information governance.
It was ironic that as I read my weekend print edition of the Wall Street Journal I spied a story (see Marriott Details Hack Exposure) they updated us all on what had been a huge data breach for Marriott before the year end. More details were being released and they were not good: some passport numbers, even unencrypted, had been exposed; same for some credit card numbers.
All in all it was grim reading. But there was one positive, albeit small, point that stood out: the analysis so far led to the conclusion that Marriott will never be able to confirm how many individual accounts had been stolen or copied.
The original number that had been shared was a massive 500 million. This has now been paired down to “an upper limit” of 383 million. But the truth is that the databases where all this data was stored has, apparently, account duplicates. In other words poor data quality (data and analytics governance practices) have allowed duplicates to persist and so any theft of individual data will not be as significant as first thought. Oh, the good news that goes with bad data quality!
Data (and analytics) governance is hot again. You often hear me on stage saying that we tend to get long running 7- or 9-year cycles where topics and ideas run hot, then cold. This is not unlike our very own hyper cycle concept. I would argue that we are four or five years ‘in’ the current cycle for this iteration of data governance. We have another two years or so to get it right, bake it in, and provide a step-change in productivity. If we get it wrong we will all have to wait for the next cycle with its new attempt at solving old persistent challenges.
What excites me most about this current cycle of interest with data governance is that after looking at the topic, 17 years at Gartner, 9 years as a vendor, and 12 years as a business user/leader, I think the newest set of best practices are top drawer. I highlight several of the main changes and themes here in a webinar: And we have written about many of these also in individual toolkits, presentations and research notes. I don’t see many organizations really nailing all these best practices at the same time, but we are getting awfully close.
Like I said, we have about two more years to make it stick. I am hopeful 2019 will be a pivotal year for data (and analytics) governance.
(This post originally appeared on Andrew White's Gartner blog, which can be viewed here).