Continue in 2 seconds

Excuse Me! Are You Getting Fresh?

  • February 28 2003, 1:00am EST

I don’t mean to be forward, really, but when it comes to data freshness for business intelligence (BI), it’s a question that really must be answered. We recently conducted a survey that found 85 percent of companies are failing to give users the best level of IT support for reporting fresh BI data.

Appfluent Technology keeps its ear to the ground by conducting a survey each month about business intelligence applications and infrastructure trends and issues. Our online surveys are completed by hundreds of business intelligence systems managers and administrators, BI application users and other senior IT and corporate managers. In January we wanted to find out exactly how quickly people want access to the freshest data, and are they getting the access they need to do their jobs well.

What we learned is that people do expect to access the freshest data to make important decisions. But, because of concerns about the cost and complexity associated with business intelligence infrastructure, companies are not fully realizing the promise of the real-time enterprise.

The survey was carried out via an online questionnaire completed by 197 U.S.-based companies across industry sectors. Three quarters of the respondents were from large or very large organizations. Participants held jobs in either business or IT management or were IT professionals. This is what we learned:


  • Only 15 percent of respondents felt that their company’s technology department supported the organization’s reporting requirements “very well.” Twenty-eight percent of respondents felt their company was only “average” in this respect.
  • Companies mainly employ one of two remedies to reporting headaches. Thirty percent of respondents run reports at off-peak times; reports are run off data warehouse or data mart systems in 17 percent of companies. Other companies offer users no solutions whatsoever.
  • One in five companies base their most crucial reports on data that is a week or more old, while the majority of companies, 42 percent, base today’s crucial reports on yesterday’s data. This number rises to 60 percent among business users who tend to run financial reports as opposed to IT managers who tend to require production reports.
  • Almost half of respondents indicated that they need fresh data several times a day.
  • Three out of four respondents list reports as being very important or extremely important to their jobs. Less than four percent said reports had no importance. For business users (54 percent), reports are more often “extremely important” than those in IT management (40 percent).
  • Overall, financial reports tend to be the most important to respondents’ jobs and e-commerce reports are seen as the least important.
  • Crucial reports take 12 days or more to execute at six percent of organizations. Almost half of respondents said they get crucial reports within a couple of hours, and one-fifth wait overnight for their reports.

It’s clear that a sizable majority of companies lack the necessary business intelligence infrastructure to meet the reporting needs of their users, and many are unaware of the problem or feel they are unable to provide a solution.
It really is a shame when you consider that last year, companies worldwide spent more than $30 billion on business intelligence software, hardware and services. Their people are not seeing the full value from that investment because they don't get access to “timely” data.Companies’ feel their hands are tied by severely constrained IT budgets. AMR, a leading research firm, estimates that companies spend $3 to $5 in services for every $1 in BI software.

The first thing companies need to do is discover what “timely” means for them. Our survey showed that, on average, 57 percent of users need information “up to the day.” Thirty percent need fresh data several times a day and two percent need fresh data “up to the second.” But these numbers vary depending on industry sector and job function.

An obvious solution would seem to be to allow the running of ad hoc queries and reports off the production database to meet the varying needs for data freshness. But DBAs know the resulting hit to application and system performance will be direct and pronounced, causing a whole different set of problems for users and IT professionals alike.

Perhaps they should consider establishing an operational data store as a mirror of the production database to relieve reporting traffic. But, that is an extremely costly and complex alternative. What companies need to do is simplify their BI infrastructure, not make it more complex, if they are to meet the needs of users.

Companies can wring the ROI out of their BI investments by delivering fresh data to users when they need it. And, with recent intelligence and automation advances in BI infrastructure, now all of these companies can afford to “get fresh.”

Next month in BI Trends, we’ll explore what tradeoffs and compromises companies are willing to make in going real time.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access