BI practitioners who preach to their business counterparts to run their business by the numbers need to eat their own dog food: run their BI environment, platforms and apps by the numbers. For example, do you know:
- How many reports and queries do end users create by themselves versus how many IT creates? That's a great efficiency metric.
- How many clicks within a dashboard does it take to find an answer to a question? That’s another great efficiency metric.
- How long does each user stay within each report? Do they just run and print the reports, or export the data to Excel, or do they really slice, dice, and analyze the information? That’s a good example of how effective your BI environment is.
- Do you see any patterns in BI usage? User by user, department by department, or line of business by line of business?
- How many reports, queries, and other objects are being used, how many are shelfware (not being used)? How often are people using the ones that are being used?
I find surprisingly little research out there on this subject, so we plan to publish a report on it. With that in mind, I’d like to solicit your input. If you have some experience, specific use cases and best practices, please share them with me. For example:
- I named a few metrics above, but I am sure we can come up with dozens more. Do you have any good examples?
- Can these metrics be organized by categories, like efficiency metrics or effectiveness metrics?
- Do you have any benchmarks that show what the targets/goals for these metrics should be?
- If you are a BI software vendor, do you build such functionality into your product?
- If you are a BI systems integrator, do you have such a service offering for your clients?
- Can you share any best practices on how to collect, organize, present, and manage these metrics?
- Do you agree that BI on BI (performance management of BI) should be a key function of BI competency centers/centers of excellence?
- I am only aware of two vendors — Appfluent and Teleran — that provide this type of a solution across multiple heterogeneous BI vendors, applications, and databases. Are there others?
I’m sure that I’ve missed some questions I should be asking — so I’d appreciate any comments and any contributions to this post.
This blog originally appeared at Forrester Research.
