Big data is all the rage at the moment. Vendors talk about it and there are a seemingly endless series of events, articles and, yes, columns about it. Everyone, it seems, is doing big data. But many of us are going about big data all wrong.

Consider the three V’s of big data that Doug Laney of Gartner first discussed way back in 2001 – volume, variety and velocity. Each of these poses a challenge for us as we try to use big data:

  • Increasing volume undermines our ability to simply display the data. Once we might reasonably have packaged up and displayed all the data that might be relevant to a decision-maker. Now there is simply too much data to do this; if we try we will simply drown people in data.
  • Increasing velocity – data arriving more quickly – is driving us to increasingly faster analysis and faster action. We have to analyze the data more quickly, give ourselves time to act on this analysis by pushing our analysis into the future, and we have to act on our analysis more quickly. This means more automated analysis, more use of predictive analytic techniques and more automated decision-making systems.
  • Increasing variety of data means we have to break ourselves of the 360-degree habit. Where once we might reasonably have aimed to assemble all the data there is about a customer into a 360-degree view, now this is simply impractical. There is always going to be another data source, more data that we could bring to bear. Waiting until we have assembled such a view means waiting forever.

The solution to these challenges is found in a six letter acronym – BWTDIM: Begin With The Decision In Mind. This re-working of the late Stephen Covey’s maxim to begin with the end in mind is a critical thinking tool for anyone trying to succeed with big data.
BWTDIM means identifying the decisions that matter to your organization, the decisions that make the difference between hitting your targets and missing them, the decisions that “move the dial.” These decisions may be strategic or tactical decisions but are also often operational, day-to-day decisions that must be made at the front lines of your organization like the call center, the branch or the warehouse. BWTDIM means understanding how you must make these decisions, and how you might make those decisions to more effectively hit those objectives. It means beginning with a model of decision-making and only then moving on to identify the analytics and data, big or little, that will be required.

Focusing on the decision first “pulls” the analytics we need. We can see what might help us make the decision more profitably and more effectively and use this to drive analytic requirements: “If only we knew how likely a supplier was to miss the ship date we could pick the right supplier.”Once we know the right analytics to use, we can see what data might help us build those analytics. Instead of assembling all the data and hoping it will improve decision-making, we assemble the data that will improve our decision-making. This decisions-first approach also broadens our view of what data to consider, helping us step outside our data warehouse by asking what data would help not what data is available.

Once we understand the decision, we can see what options might exist to automate some or all of it. It’s easy to say that a decision must involve a person because it’s too difficult to fully automate it. With a focus on the decision and a model of how that decision should be made, we can see if there are pieces of the decision that can be automated. We can move from presenting data to decision-makers to presenting answers and recommendations.

A focus on using analytics to make decisions rather than on presenting data for analysis means we don’t have to display all the data. Instead of making it all available for a decision-maker to use the best he or she can to deliver a decision, we can at least present a strongly decision-centric analysis instead. And this focus also makes it clear what the role of data mining and predictive analytics will be. Understanding how we would like to make a decision and what it would take to improve it allows us to identify clear roles and value propositions for these more advanced analytics.

BWTDIM – Begin With The Decision In Mind – offers a clear approach to the challenges of big data. After all, we know that big data and analytics are all about improving decision-making. If we really want to improve decision-making we should begin by defining the decisions we want to improve.