Business intelligence (BI) is booming. Everyone I talk to these days is undertaking or well into a major BI initiative. Enterprise reporting is a product that clearly has been needed, and the BI vendors have responded with enterprise class tools to support enterprise reporting. In the gold rush to provide businesses with dashboards sporting the latest eye catching visualizations, the accuracy of the information that feeds the BI tools almost seems like an afterthought. Do your metrics accurately reflect the events they are measuring? Is your actionable information accurately reflecting your customer, ordering, and product transactions? Let me illustrate a recent interaction with my wireless provider to highlight how your company may be misinterpreting business activities.

 

In my wife’s company newsletter, our wireless service provider had an advertisement that offered a ten percent discount off of our monthly service charge just for being an employee. Great news, I thought! We’re already customers.

 

Well as we all know by now, business rules, and the applications that are built to support those rules, like events to occur in a nice, contiguous manner. The business rule I ran into is my wife needed to be the “primary” customer on the account. Since I originally set the account up, I was the primary and that was end of the discussion. “Make my wife the primary,” I retorted.

 

Here is where things went south quickly. In order to change my wife to the “primary” on the account, our wireless provider closed our old account, and created an entirely new account complete with a new account identifier.

 

Telecom companies put a lot of thought into how to attract new customers as well as keep the ones they have. Promotions are a great way to get new business in the door. Once inside the company, cross-selling other products and services improves the likelihood of you remaining a customer. And I assume that the marketing folks want supporting metrics from the data warehouse to validate how successfully a promotion is or is not working. So what is the answer to the question, am I a new customer? If you take a holistic view of the transactions across my two accounts, you should say no. But if you’re looking at new install order transactions generated by my primary account change request, I bet I look just like a new customer! You would also come to the same conclusion by looking at the provisioning and billing transactions. Even my wireless provider, with whom I have logged five faithful years of service, has lost insight to my previous identity. Another opportunity for customer data integration rears its head.

 

The folks in the data warehouse are faithfully capturing all of these events to support their key performance metrics related to a given promotions effect on customer installs and disconnects. And based on my experience, I imagine my transaction will dutifully go down as a successful new customer install. But my install was not a success. It should not have been counted as an install at all. At best, it was a wash. There might not be a statistically significant sample of customers like myself, but other scenarios like these exist in every company. A business event is incorrectly categorized due to lack of business or operational application expertise in the warehousing team.

 

To exacerbate matters, public companies publish customer metrics to Wall Street to show how they are improving or possibly losing market share. If SOX compliance isn’t enough to scare you, the Wall Street Journal recently reported that Verizon is being sued for allegedly reporting inflated subscriber numbers for their fiber optic services (FIOS).1 Would you be prepared for the intense scrutiny a lawsuit would bring to correctly account for each new customer in your data warehouse? And, for the record, Verizon is not my wireless provider.

 

Simply validating transactions is not enough. Misinterpreting business events can undermine the validity of the transactions that make up your key performance indicators (KPIs) and other business performance metrics. Here are a few ideas to ensure your metrics don’t lie.

 

Engage the Application Experts

 

Trying to reverse-engineer business rules from data profiling and SQL analysis is an impossible task. But that is just what happens on many data warehousing teams. Instead of trying to derive knowledge from data profiling reports, reach out to the in-house application experts. These are the people who toiled many long nights and weekends to implement the applications that run your business. They are the people that should be engaged on any business intelligence project to ensure you are interpreting the transactional events in the data correctly.

 

Reconcile to the Source

 

Too often, testing takes a back seat in warehouse projects. The requirements are gathered, the source data is profiled, the target data model is built, the extract, transform and load (ETL) is written, and the BI layer is developed. After a cursory unit test by the warehouse team, the duty of in-depth testing of the BI application is left to the business. After all, this is self-service reporting! Do not rely on or even expect the business to perform comprehensive testing of new BI reports. All reports should be reconciled back to the online transaction processing (OLTP) database by the warehouse team. This isn’t a one-shot test either. Implement an ongoing monitoring process to track how data quality is improving or worsening. In my experience, organizations with dedicated testing teams deliver higher quality reporting applications. If your organization has such a team, you’re on the right track for keeping your metrics honest.

 

Kick off a Data Quality Program

 

In a recent white paper on data quality, Robert Lerner summarizes, “Organizations should begin a data quality program, one that includes technology as well as processes, data governance and best practices, with the idea that it will support all of their data.”2

 

The accuracy and quality of your metrics fall under the domain of the data governance organization. The data governance organization is a key ally for the warehouse team to leverage business expertise within their organization. Only with business and IT working collaboratively can you begin to uncover the true breadth and complexity of discrete business events.

 

References:

  1. Christopher Rhoads. "It's Verizon's New Fiber Optic Network that Delivers TV and Broadband Internet Service." Wall Street Journal, October 3, 2007.
  2. Robert Lerner. "Data Quality by the Numbers: Best Practices for Managing Business Data." Dataflux.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access