Continue in 2 seconds

A Rational Approach to Chargeback for Data Warehousing, Part 2

  • March 16 2006, 1:00am EST

Last month, this column looked at the shocking case of a data warehousing client whose toughest business intelligence (BI) problem was chargeback. You might think that this individual needed to get out more or find something useful to do. However, upon more careful inspection, this person's firm presented a problem that many enterprises find so intractable that they have given up even trying to deal with it. Last month we looked at the differences between the traditional data center environments and those typical of business intelligence or exploratory data warehousing. We also looked at several principles on which to base a rational chargeback system. These included guiding principles such as centralization; distinguishing between cost and value; and avoiding chargeback systems that discouraged use of the data warehousing resource, that was more complex than the system being metered, or that typically resulted in billing disputes. This month we implement those principles and consider several alternative chargeback systems that avoid detailed metering of CPU cycles, I/O or SQL inquiry statements, and still capture business value.

Alternative Methods of Performing Chargeback

Let's go right to the heart of the matter. Three alternative methods to chargeback include weighted customer and account pricing, pure user-based pricing and ASP-like pricing. Let's look at these in turn.

Weighted Count of Customers and Accounts

The idea is that the number of customers queried and accounts handled per department are a good approximation of the value derived by the respective department functions. At startup and periodically thereafter, the number of customers and the number of accounts are counted. The departments are "charged back" pennies per account and pennies per customer under management as determined at startup and periodically (annually) thereafter. A flat initiation fee is also assessed.

In general, some departments have many customer-based projects, campaigns and queries whereas others have relatively more account-based projects. In financial services, an account is often a good proxy for a product. Some departments are customer focused whereas others are product focused. This method balances the two by taking a weighted proportion of each as part of the total system cost to the department. So, for example, a given department pays one-tenth of a dollar per customer and one-tenth of a dollar per account multiplied by a customer/account weighted model of 40/60. The customer fees are multiplied by 40 percent and the account fees are multiplied by 60 percent. In one specific, real-world example where this method was successfully implemented, a particular enterprise combined a cost allocation based on a shared customer infrastructure with an initial up front fee to participate. Therefore, no matter how small or large your line of business unit, it cost $500K per year to use the BI system.

This does not require a given department to "own" a given customer or account, though this approach can also work. It merely requires departments to access and inquire against customers and accounts. It is not even necessary to have a single, centralized representation of customers and accounts, though coherent master data management is a best practice that aligns well with this approach. However, if a department has a "secret" customer master file to which the centralized data administration function has no visibility, this method will not capture the value derived from using that data. In the limit case, though highly unlikely, if all departments accessed all customers and all transactions, then all would pay the same (maximum) fee. This would be justified by value-based pricing. All are getting the maximum benefit - customer access, opportunity and interaction.

This works best if a simple SQL count of customers and accounts against a department ID can be executed. However, that is not an absolute requirement if multiple departments are launching ad hoc queries against an overlapping set of customers or accounts. A prepackaged mini-audit can be defined and performed to determine the volume of accounts and customers that are touched by a given department in the course of its queries, campaigns, projects and initiatives.

It is theoretically possible that a department might simply not know what accounts or customers it touches. In this case, a mini-audit looking at a sample of the inquires executed should give a reasonable starting profile. This can then be updated periodically (yearly) and iteratively based on usual and customary activity.

Pure User-Based Pricing

This is a proven model for establishing value in both end-user computing and server-based configurations. Figure 1 is a hypothetical example for discussion.

Figure 1

The license could be either term or perpetual. If this is a term license, then it will be renewable periodically (perhaps annually). It will be similar to renting the software. If the license is perpetual, it will be one large up front fee with annual maintenance (perhaps at 20 percent). The latter would be a better choice if the enterprise is looking to cover the costs of development, because that could be priced to include one large up front fee. However, if development is separately handled, then the term approach has the merit of being relatively simple and avoiding the hassles of a long-term ("perpetual") contract.

With this approach, a third alternative is available for licensing. Under this scenario, a site license for all staff in an entire department (or corporate grouping) is also available with the specific terms being negotiable and, presumably, allowing for size-based discounting.

ASP-Like Pricing

Although I am not aware of any enterprises using the application service provider (ASP) model for chargeback, it is one well-suited to that purpose, and it is surprising that more have not tried it. Typically there is an up front fee to handle setup and administrative costs. Then there is a per user fee per month. Finally, some ASPs charge for the amount of data storage utilized. This is designed to incent users to clean up after themselves. So it actually does discourage use of the resource; however, in this case, that is not considered a disadvantage but a desirable feature.

If the ASP is operated as a single, shared resources where, for example, the entire installation has access for exploratory purposes to the entire set of transactions, customers or accounts, then charging separately for storage will not be appropriate. It is a cost to be factored into the monthly service charge. However, if users are building (proliferating) data marts, analytic applications or intermediate proprietary data sets within the centralized (administered) system, such a storage charge may usefully be made a part of the mix.

A couple of additional conditions and qualifications should be noted about all three of these methods. An effective chargeback system for business intelligence or related ad hoc exploratory ("research") data warehousing does not eliminate the need for a capacity planning team and sound capacity planning. Capacity planning is still on the critical path for maintaining service level agreements over the long term and optimizing performance and tuning. It merely means the administration of chargeback is simplified and eliminated as a source of stress.

An effective chargeback system for BI is different from defining and implementing the service level agreement (SLA) supporting the line of business units in using the BI system. Often, an SLA accompanies signing up to use a shared resource and represents an important part of the agreement describing what is supposed to happen if the resource is unavailable, if the user experiences a breakdown in his or her use of the service or simply requests support. Strictly speaking, an SLA is different than a chargeback system; but a fair and effective SLA can go a long way in keeping business units happy with whatever method of chargeback is employed.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access