OCDQ Blog
for Information Management Blogs
NOV 18, 2011 12:41pm ET

Blogroll

The Speed of Decision

Print
Reprints
Email

In a previous post, I used the Large Hadron Collider as a metaphor for big data and big analytics where the creative destruction caused by high-velocity collisions of large volumes of varying data attempt to reveal elementary particles of business intelligence.

Since recent scientific experiments have sparked discussion about the possibility of exceeding the speed of light, in this blog post I examine whether it’s possible to exceed the speed of decision (i.e., the constraints that time puts on data-driven decision making).

Is Decision Speed more important than Data Quality?

In my blog post “Thaler’s Apples and Data Quality Oranges,” I explained how time-inconsistent data quality preferences within business intelligence reflect the reality that with the speed at which things change these days, more near-real-time operational business decisions are required, which sometimes makes decision speed more important than data quality.

Even though advancements in computational power, network bandwidth, parallel processing frameworks (e.g., MapReduce), scalable and distributed models (e.g., cloud computing), and other techniques (e.g., in-memory computing) are making real-time data-driven decisions more technologically possible than ever before, as I explained in my blog post "Satisficing Data Quality," data-driven decision making often has to contend with the practical trade-offs between correct answers and timely answers.

Although we can’t afford to completely sacrifice data quality for faster business decisions, and obviously high quality data is preferable to poor quality data, less than perfect data quality can't be used as an excuse to delay making a critical decision.

Is Decision Speed more important than Decision Quality?

The increasing demand for real-time data-driven decisions is not only requiring us to re-evaluate our data quality thresholds. In my blog post “The Circle of Quality,” I explained the connection between data quality and decision quality, and how result quality trumps them both because an organization’s success is measured by the quality of the business results it produces.

Again, with the speed at which the business world now changes, the reality is that the fear of making a mistake can not be used as an excuse to delay making a critical decision, which sometimes makes decision speed more important than decision quality.

“Fail faster” has long been hailed as the mantra of business innovation. It’s not because failure is a laudable business goal, but instead because the faster you can identify your mistakes, the faster you can correct your mistakes. Of course this requires that you are actually willing to admit you made a mistake.

(As an aside, I often wonder what’s more difficult for an organization to admit: poor data quality or poor decision quality?)

Although good decisions are obviously preferable to bad decisions, we have to acknowledge the fragility of our knowledge and accept that mistake-driven learning is an essential element of efficient and effective data-driven decision making.

Although the speed of decision is not the same type of constant as the speed of light, in our constantly changing business world, the speed of decision represents the constant demand for good-enough data for fast-enough decisions.

How quickly do you decide? Please share your thoughts on how decision speed affects data quality, business intelligence, and data-driven decision making.

This post originally appeared at OCDQ Blog.

Advertisement

Comments (2)
I think the relationship between decision speed and data quality needs to consider many other variables beyond speed; eg decision scope, time horizon, consequences of being incorrect etc. I agree organizational ability to recognize and adjust to errors is an important factor. An incorrect decision based on poor data could still get you to the best answer sooner if a constant continuous improvement mindset (as you point out, fail fast but learn). Individuals, especially children, operate that way but it is much more difficult for organizations. Organizations need to see the entire cycle, from data to decision, in order to find and address the constraint to being faster and better decision makers. Cliche, but I think the issues this blog raises really illustrate that partnership across functions, connecting the white space, asking "when" instead of yes or no questions, is important to getting data driven decision making, BI and data quality to the next level.
Posted by Ed U | Sunday, November 20 2011 at 6:22PM ET
Thanks for your excellent comment, Ed. Especially your point, "an incorrect decision based on poor data could still get you to the best answer sooner if a constant continuous improvement mindset" is being used. I believe that the biggest obstacle to taking data management and business intelligence to the next level is that the organization is inherently biased against believing that poor data quality and poor decision quality are prevalent. Instead of a constant continuous improvement mindset, most organizations prefer a constant continuous blissful ignorance mindset. Which is why so many organizations complain when they are blindsided by data management disasters and business intelligence blunders. Although the business world will never be totally predictable, we can not turn a blind eye to either the need for data management and business intelligence best practices, or the reality that no best practice exists that can completely eliminate the potential for poor data quality and poor decision quality. A key concept of statistical process control and continuous improvement is the importance of closing the feedback loop that allows a process to monitor itself, learn from its mistakes, and adjust when necessary. The importance of building feedback loops into our decision making is too often ignored. Best Regards, Jim
Posted by James H | Monday, November 21 2011 at 9:52AM ET
Add Your Comments:
You must be registered to post a comment.
Not Registered?
You must be registered to post a comment. Click here to register.
Already registered? Log in here
Please note you must now log in with your email address and password.

Blog Archive for Jim Harris

Data Cleansing is a Life Saver
Fail Faster Flaws
Data Quality: Get Ahead in the Count
First Pitch Toward Data Quality
Data Science from a Different Angle

More from Jim Harris »

Blog Index »

Twitter
Facebook
LinkedIn
Login  |  My Account  |  White Papers  |  Web Seminars  |  Events |  Newsletters |  eBooks
FOLLOW US
Please note you must now log in with your email address and password.