Living with Imperfect Data

In a keynote at our MDM & Data Governance conference in Toronto a few days ago, an executive from a large analytical software company said something interesting that stuck with me. I am paraphrasing from memory, but it was very much to the effect of, “Sometimes it’s better to have everyone agreeing on numbers that aren’t entirely accurate than having everyone off doing their own numbers.”

Let that sink in for a moment.

After I did, the very idea of this comment struck me at a few levels. It might have the same effect on you.

In one sense, admitting there is an acceptable level of shared inaccuracy is anathema to the way we like to describe data governance. It was especially so at a MDM-centric conference where people are pretty single-minded about what constitutes “truth.”

As a decision support philosophy, it wouldn’t fly at a health care conference.

The flip side is that this statement also happens to be a pretty honest admission of the way people get things done collaboratively – and the downside that might come from a metric defined elsewhere that can affect broad parts of an organization.

It’s acknowledging in part what Boris Evelson was saying last week about IT reconsidering its control philosophy, and also about what’s been said about the boundaries of governance.

It’s not necessarily an attribute of data to be right or wrong; often, data just is what it is. But how do we live with decisions formed from conclusions where the numbers we use are not numbers we’ve agreed to?

I couldn’t wait to bring the topic up later that day in a panel I was hosting on data governance where I had a couple of real veterans, Ed Unrau from Canadian Tire and Chai Lam from Bank of Montreal to help me lay out the problem.

Ed said the question is a challenge of true enterprise data management, but he bisected what we were talking about, correctly I think.

“I distinguish data from information so we can talk on one hand about master data management, but you can also think of the reporting that comes from that data and call that information and information governance,” he said. “Let’s say we want to be more customer centric, we’ve created this repository of data and really smart people analyze it, but if we have three different customer segmentations and different versions of how many customers we have going up or down in a region, that is going to be a big problem for the entire customer centric strategy.”

That implies guardrails, the rights and ownership at the heart of governance – and governance is something Ed said should be about enabling and not restricting widespread data use. What you want, I figured, is optimum use of the data and you probably do that by summoning the least inconsistency you can muster.

Ed picked up on that, preferring the word “inconsistency” to “incorrect,” and reminding us that you don’t want to restrict data in absolute terms for the sake of either word. In a compromised world where things are always going to be damaged overall, “in some cases you can react to consistently incorrect information, you know where to steer and what to do. If you instead have inconsistent information all over the place, VPs throwing different numbers back and forth about whose fault something is, that is a big problem.”

It was a little bit liberating to hear this in a crowd of people who, in their role of advocating governance, are usually out to think in terms of things that are either acceptable or unacceptable.

Chai Lam from Bank of Montreal  picked up on the small versus enterprise theme. “If your organization believes in governing a set of data in a line of business and that is where executive sponsorship is going, there is no mandate to make it enterprise. You know IT is there to enable them, but you have to make sure you have a sponsor in your line who mandates the scope.”

His larger point was about stewardship as well as governance and that they are not the same thing. “Governance is all about policy and standards. Stewardship is the operationalizing of all these policies and standards on a day to day basis. Somebody has to look at the data and say, ‘I have two duplicate profiles, who makes the decision to collapse them?  What are the right business rules to apply and clean it up?.’”

At one point, the discussion took a technical turn into data extensions, and Ed replied to a comment to say, “That is such an IT answer!” 

And that’s the point, Chai said. “You have to have the business processes and the organization and the people to do stewardship work and it’s not IT work, it’s up to business to operationalize this. You can’t have governance without stewardship.”

There it was. We’d accepted the frailty of a single version of truth at, of all places, an MDM conference.

People have to have confidence in foundational data, even if we’re not defining it as right or wrong. Different consumers of data will have different ideas about rules. They will suffer or benefit more or less based on the consistency of the data because we all live with imperfect and inconsistent data and don’t usually get to define all of it.

An audience member had the last word: “Ideally you want to get it right, but if everybody is held to a single source and they don’t feel it’s right, they are going to walk away and do whatever they think needs to be done.”

As one of our 2012 25 Top Information Managers reminded me recently, humility really does qualify as a leadership skill.