When I was a junior programmer I was encouraged to make applications "table-driven" as much as possible. What my project managers meant by this was to resist placing data values in program source code, and instead to have them in database tables. This made the applications more flexible. For instance, some source code might implement a business rule for calculating taxes on retail sales using a tax rate of three percent. If three percent was implemented directly in the source code, then if the rate changed to four percent, all the source code using three percent would have to be found. The programs concerned would have to be changed, and the new programs would have to be implemented in production. By contrast having the three percent in a database table that was accessed by the appropriate business rules meant that there was only one, known place where the data item needed to be changed. The flexibility of the table-driven approach supported ease of production maintenance, rather than anything else. However, it did mean that the defined components of a single business rule were found in more than one place.

Nobody that I know of was thinking about business rules in the late 1970s when I started programming. Yet, I did find that the table-driven approach evolved to become more sophisticated than simple placing known data values outside of program source code. A variant of it consisted of triggering particular pieces of source code based on data values in records being processed. This meant that programs could be written that included a large set of rules. For instance, a program to calculate gross payroll would have one set of rules for a married person and a different set of rules for a single person. The appropriate rules would be triggered by the code for marital status on the person record. In other words, it was possible to write business rules for many different scenarios and trigger the appropriate ones on demand.

The Consequences of the Table-Driven Approach

All of this seems blindingly obvious, and indeed it is from the perspective of a programmer. However, it has rather profound implications for business rules. The first is that a single business rule may have a definition that includes constructs in a database, and constructs that are held somewhere else. We talk about the need to keep business rules atomic, and to define them in one place. Yet, most IT professionals would feel uncomfortable about having a data item like three percent held directly in the definition of a business rule rather than in a database table. After all, the data item might need to be used in other contexts, e.g., on reports. Placing data items like this in a database table means that the physical boundaries of a business rules may be rather nebulous. For instance, a business rules may be implemented as meta data, or even as source code, but to understand it fully we must also look at data values held in database tables. A business rule may be defined once, but it may consist of two major parts - something that represents logic and something that represents preassigned data values. What is more, these data values are nearly always reference data.

Reference Data

Reference data is often called domain values or lookup tables. It is usually the database tables that consist of a single code column and a single description column and a few rows. Examples include, country, currency, customer type, product category, order status, and so on. Sometimes, these tables include additional columns like tax rates. What is amazing about business rules is that when an actual data value is included in the rule, it is nearly always something that comes from a reference data table. Yet there is very little appreciation of this tremendous linkage either in the current thinking about business rules or the products offered by vendors.

The examples of the table-driven approach given above are rather simple and do not really do justice to the key role placed by reference data in driving business rules. Perhaps a better example is where reference data is used to control what I call "hidden subtypes" in the execution of sets of business rules. Any table such as customer is often related to many reference data tables. These tables act to identify subsets of records in customer, such as all customers with a preferred status of "gold" and who are located in "Canada" and whose primary business is "finance." For this particular subset of customer there may be some special business rules that we need to apply. Reference data enables us to identify the appropriate records. There are reference data tables for preferred status, country and business sector that are related to customer. This contrasts with creating formal subtypes, which would require us to create one additional database table for each identified subtype. Formal subtyping is rather rare in database designs, one reason being that it is difficult to program around. Hidden subtypes, by contrast, mean that all customer records are in one database table, where the subtypes are identified by using reference data values.

The Issues of Linking to Reference Data

There are many profound implications of the fact that business rules need to link to reference data values. A major one is that reference data changes over time. Suppose a customer had a preferred status of "gold" last month, but a preferred status of "silver" this month. We may have a business rule that states that a "gold" customer is rebated one percent of their monthly balance at the end of each month. The logic of this rule may not have changed in the part five years, but the preferred status of the particular customer we are looking at has changed in the past month. It makes no sense to see that the currently silver customer had a one percent rebate last month. In other words, we may go to great lengths to record changes and keep versions of the logic of business rules, but may have no equivalent change control for the related reference data. The result is that the current set of business rules have no relation to the historical values that they supposedly calculated.

Reference data and business rules have been linked for decades. It may have seemed that it was initially for programming convenience, but in reality the linkages occur on many levels and have deep ramifications. These ramifications are very poorly understood, even today. Until the methodologies that are being developed to deal with business rules recognize the role of reference data, we will never have a proper conceptual grasp of how to manage the rules. Until vendors start supplying tools that properly interact with reference data, business rules solutions will always have shortcomings.