What is going on in the world of data and analytics?
I have blogged before (see this from 2014: A Day in the Life of an Analyst” at Gartner’s IT/Expo Symposium – Day 3) about the hot topics I discussed with attendees at our Symposia and data and analytics conferences. In the last 18 months I have not kept this effort up since the observations I had been making were not changing that much. This has as much to do with a slow changing industry as it has with a slow changing attendance profile.
All this changed this year. After three weeks of conferencing around the globe (in the last 5 weeks), my mind blew open today (yesterday, when I wrote these notes) and the following is what emerged.
There are several threads and messages that have emerged clearly or adjusted from past trends. All these are notable but two were significant, at least for me. The notable ones include (in no particular order of importance:
- The time of business information architecture is now. MDM is dead; at least the old idea that was peddled that all data in a master file is master data. MDM is far from dead – but we need a modem approach to MDM (so we need a new name?) to weave together the governance and management of master data, application data, and less-widely shared data, and just enough enterprise metadata management.
- Chief Data Officers (CDOs), D&A leaders and their offices that own delivery of data and/or analytics struggle to perform compared to CDO (and D&A leaders) that don’t. This difference in scope often explains the vast difference in size of resources in the CDO team/office.
- Economics over architecture. I jest here; what I mean to say is that economics and rational, smart models should trump perfectly designed ideas. Another way to say this is modern architecture is lean architecture.
- Age maybe against us. Ted Friedman spotted this before I did. Many of us, attendees and clients, are getting on in years and some are nearing retirement. Plan for senior leadership change….
- To Centralize or decentralize- that is the question. Well, it depends on the work in scope…data management, or analytics, or data science/ML, or data (and analytics) governance….
- CDO role is gaining traction; though some are still not quite ‘chiefs’
- Chief Data and Chief Digital officer roles are getting closer than ever in terms of scope and responsibility; digital officers are more concerned with being able to effectively fulfill their digital designs; data officers are much less focused on technology and more focused on the least amount of information that maximizes business outcomes.
- Data and analytics (and maybe AI) are getting very tired terms. Maybe we can explore things like decision making?
- Metadata and its management across an organization is becoming critical (and this is not the same as ‘metadata management’). Yes, it is more important than ever but for all our sake, don’t start any conversation with this topic.
Now for the two (for me) really notable observations. The two specific questions from different attendees that encapsulated the breadth of change going on were:
- How and where can I automate the work of data (and analytics) governance?”
- How can I account for the value we generate when we are organized as a cost center?
Let’s start with the first. This is a simply put question that masks a lifetime of complexity and dissatisfaction. After all, analytics, BI, and data science/ML platforms have their trajectory and path and number vendors and sectors collide, acquire, integrate and automate. We have our own magic quadrants and market guides exploring this market.
This is pretty similar story to the parallel world of data management; lots of platforms, lots of tools; acquisitions and so on. There is increasing overlap between data management and analytics. But what of the work of data and analytics governance? It remains pretty much a mess. Why is this the case?
Some few years ago a colleague of ours, Deb Logon, mused on the idea of a data (and analytics) governance platform. It made so much sense; it was logical and simple to understand. She conceived the idea that all the needed tech-enabled capability ought to converge into a single platform. This platform would complement the already established data management platform and the analytic/BI and data science/ML platforms.
Deb’s idea was too early but it played a key role in describing our collective future. As time passed, we observed an intermediate formation of a set of those capabilities. In fact I predicted this initial early formation with a screen mock-up of such an application many year ago at one of our D&A conferences. But to describe this intermediate formation we had to discover and describe the work of data (and analytics) governance in such a way as to separate what a business leader, a business user and an IT person does.
We did this a few years ago and we have described this (see The Case for Stewardship versus Governance) as:
- Policy Setting
- Policy Enforcement/interpretation
- Policy Execution & Data Changes/Entry
Our colleague Mark Beyer agrees with the view and gladly places the world of data management in the third layer – this is where policies are executed. With this framework we were able to describe the initial formation as ‘what a business person needs to use to solve business problems, processes and decisions held hostage to data’. The short name is ‘information stewardship solutions and it is where policies (there are 8 of them) are interpreted and enforced, in support of a business outcome.
These early solutions emerged with business oriented user interface to help business users solve problems; this was dramatically different from the years of development of metadata management tools for IT. This new sector emerged a few years ago and we introduced a market guide. Clients started to recognize the solution and vendors started to see the opportunity. All was going well. Then GDPR happened.
About three or four years ago GDPR started to become an imperative. The result was that both end users (non IT companies) in public and private organizations, and vendors, changed direction: clients looked for quick fixes to help avoid risk; vendors refocused away from their current trajectory of developing solid information stewardship solutions toward the ‘easy money’ of GDPR. All the while the MDM vendors, where slowly waking up to the need to support policy enforcement and setting, continued their focus “down” toward the data management and execution work.
So here we are. The evolution of information stewardship has both stalled due to GDPR demand and also evolved due to noise across the different data (and analytics) governance policy sectors (such as data security, data privacy, ethics etc.).
Some information stewardship solutions do also offer some various support for policy setting; some use the words data governance when in fact they mean information stewardship. Some don’t understand the difference. Some do and deliberately they will use a different word. Too many vendors have convinced their clients they have information stewards when fact they have more data quality analysts. We are again in a more complex space then we were 5 years ago.
Yet here was a client, sitting in front of me, asking one question that blew my mind since it wrapped up the last 12-15 years’ work and observations in a few words:
- Where is the platform that provides all the (integrated) capability to govern data, and analytics,
- Across all policies such as security, privacy, quality, retention, availability etc.
- Across all kinds of data such as structured and unstructured, analytics, algorithms, information, documents, images, and so on.
- That helps automate as much of the work (policy setting, policy enforcement and policy execution) as possible?
It is a great question. It’s a question I hope I see more of. But there is no such platform. There won’t be such a platform for a number of years. There are some acquisitions that look promising; there are few mega-vendors with compelling visions, let along funded road maps or executives with P&L responsibility in this area.
We are now stuck in second gear; stuck in a period where we know how to add value to many business outcomes with effective and modern data governance efforts with information stewardship solutions. However the market – users and vendors- are out of alignment and focused on very specific issues like privacy rather than a broad-based approach.
The good news more and more organizations are adding the role information steward, but they are not asking for the right solutions of the vendors to help them do their job. Demand and supply is misaligned. This will keep us busier than ever for longer than I had hoped.
Now for the second really notable observation: How can I account for the value we generate when we are organized as a cost center? This again blew my mind. One simple question highlighted a fundamental issue we face in the industry. So much of D&A, many CDOs or their offices, are set up, organized and managed as a cost center.
This does not make any sense given that data is not a cost of doing business; it is part of what describes the business – and for some organizations it IS the business. Analytics is what is used to inform a decision, even if the decision making does not look at a newly minted dashboard – they are still using data and analytics. D&A has more to do with business than technology!
But the “how do I show value” question goes to the center of the indirect nature by which all things data, analytics and AI add that value. It is hard to explain and express and measure that contribution. But there are a growing number of ways to do so. But what about that cost center angle?
The conclusion is that CDOs need to set up their offices up as a P&L, not a cost center. CDO’s need to start setting those expectations and leveraging data literacy to get there.
This idea of behind a P&L and not a cost center goes to the heart of:
- How value-add is measured and recognized
- How the organizations’ capital is employed – see Where You Spend Your Firms’ Capital Matters
- Where the CDO reports
This is certainly a Gordian knot for many CDOs, D&A leaders, and organizations in general. It we need to figure this one out. If we don’t, the role will struggle to survive as a discreet role. So now it’s time to go home and rest, and let these ideas percolate. This is why I love my job – we get to talk to hundreds of organizations, big and small, public and private, from all around the world. We get to listen, learn, think, and help. And we get paid to have the most fun with learning all the time. I recommend it!
(This post originally appeared on Andrew White's Gartner blog, which can be viewed here).