A few weeks ago I wrote a blog critical of the EMC survey/report “Data Science Revealed: A Data-Driven Glimpse into the Burgeoning New Field.”

Among my concerns was an uncertainty of just how representative the sample of 497 was of the population it purported to describe. I was also skeptical of several trends and conclusions drawn from the limited data. In short, I thought the research might well be biased in the direction of “proving” that data science was growing and hot, while business intelligence was declining and not.

To show I’m not just a curmudgeonly naysayer, I’d like to acknowledge an alternative series of studies I think are especially well done and would offer as a standard in the analytics space.

Analytics: The Widening Divide” is the 2011 follow-up to the 2010 “Analytics: The New Path to Value” that I wrote on last year. The studies certainly have pedigree: The research is sponsored by the prestigious New Intelligent Enterprise, a joint MIT Sloan Management Review and IBM Institute of Business Value analytics research partnership.

In the 2010 survey, over 3,000 executives, managers and analysts representing 108 countries and 30 industries completed the 22 item questionnaire. In 2011, more than 4,500 managers and executives were interviewed. These numbers are almost an order of magnitude greater than those from a standard Web-based BI survey. In the absence of random sampling, the volume and the industry/geographic coverage give me at least some comfort that the sample is representative of the population of senior analytics practitioners and executives. I rarely feel so confident after perusing results of standard BI “research.”

There are three major question groups in each of the NIE surveys. The first covers basic information such as industry, respondent role, geography and revenue. The second revolves on analytics, with specific questions probing business challenges, specific uses of analytics, perceived value from analytics, and obstacles to analytics deployment. Third, there are a number of Likert-like scales that assess the organization's overall performance, as well as its sophistication with information-consumption and analytics. Much of the reporting focuses on correlating variables from each of the three question groups and noting trends from 2010 to 2011.

Both reports distinguish companies by their levels of analytical sophistication. Aspirational organizations have basic spreadsheet users generally consigned to analytics in finance and supply chain management. Experienced organizations are comprised of more savvy analytics practitioners who use a portfolio of analytics tools to guide strategy, marketing and operations. The most sophisticated, Transformed organizations, are characterized by strong analytics proponents who use advanced modeling tools “to guide decision making in day-today operations and future strategies across the organization.” In 2011, 72% of Transformed respondents were obsessed with speed in decision-making, compared with 49% of Experienced but only 22% of Aspirational.

“Analytics: The Widening Divide,” not surprisingly, emphasizes changes in analytics sophistication noted from 2010 to 2011. “Our previous study showed the emerging gap between organizations that use analytics for competitive advantage and those that do not. This year, we see a divide that is even larger, and is rapidly widening.” Indeed, 80% of 2011 Transformed respondents noted an increasing competitive advantage from analytics compared to 65% in 2010. The numbers were even starker for Experienced users: 83% versus 38%.

"The Widening Divide" articulates three core competencies organizations must master to achieve a competitive advantage with analytics. The first is information management, which focuses on standardized data practices. The second is analytics skills, which revolve on core discipline expertise built on robust tools. Finally, there’s a data-oriented culture that sees analytics as a key asset to drive evidence-based management.

Transformed organization culture is open to new ideas with insights available to those that need them. These organizations demonstrate that analytics is core to business strategy and operations by making significant investments in both managing and analyzing data. Their end game is to embed predictive analytics into business processes.

I recommend readers investigate both the 2010 and 2011 NIE reports. You’ll no doubt learn a great deal about the direction of analytics in business from your efforts. At the same time, you’ll probably raise your personal expectation bar for what constitutes quality with BI/analytics research.