This is unsurprising, since the dimensions of a data warehouse are essentially master data (e.g., hierarchies of products, customers, locations, etc.). However, despite the close relationship between MDM and data warehousing, a glance at the recent literature reveals that these two important areas tend to be treated as separate.
At The Information Difference, we have conducted a survey focused on exploring the linkage between master data and data warehouses. The survey also aims to help us understand the scale, scope and success rates of MDM and data warehousing initiatives in business.
There were 208 respondents drawn from a wide spectrum of industries all around the world that completed the survey; the majority were from North America (57 percent) and Europe (27 percent). More than half the respondents (53 percent) came from companies having annual revenues greater than $1 billion. Some of the key findings are outlined in this article.
Many Organizations Already Have both Data Warehouses and MDM
Respondents to the survey were asked which of the classifications shown in Figure 1 best represented the position in their organization.
Almost half (46 percent) of the respondents had at least one data warehouse and an MDM program either live or in development, showing the growing adoption of MDM compared to our earlier surveys. Encouragingly, the majority (57 percent) of MDM implementations are enterprise-wide. MDM implementations generally manage a median of 4 million records.
MDM Data Domains are Wider than Customer and Product
Although the common master data domains of customer and product received the highest priority and were most often selected by the respondents (see chart below), the average number of domains covered by organizations is 4 (with a median of 3). Location and supplier closely followed customer and product in importance according to the respondents. (Note: The figure shows the relative number of times each domain was selected by the respondents with the numbers normalized for the total number of responses. Essentially, the higher the score the more often the respondents selected the master data domain in question. The scale on the chart represents the normalized number of selections and not a percentage. )
This shows that while customer and product still have high priority, most organizations have a need to cover a broader range of MDM data domains. Encouragingly, fully two-thirds have data governance in place showing an improvement over our earlier studies. Importantly, almost two-thirds of organizations are feeding their master data directly to the BI applications. Indeed, this is supported by a strongly held view (81 percent) that master data should be sourced directly from the MDM system; some 36 percent were already doing so.
Encouragingly, of those who have already implemented MDM, the dimensional data in the warehouse (essentially the master data) is overwhelmingly maintained in their MDM system.
Organizations Regard MDM Initiatives as Successful
Two-thirds of respondents consider their MDM implementation to be at least moderately successful, with only 1 percent rating it as unsuccessful.
Only 27 percent of organizations reported that they had a single data warehouse. The median was three warehouses, with a median of 10 source systems underlining the complexity of most business data warehousing operations.
Scope of Data Warehouses is Generally Enterprise Wide
The scope of the data warehouse implementations for three-quarters of organizations is enterprise-wide with 23 percent being “departmental” as shown in the chart below.
Organizations in general support a mean of 3,500 users (median 200) with some 28 full-time employees for maintenance (median 10 FTEs).
The biggest data warehouses are around 500 to 800 terabytes with a mean of 34 and median of 4 terabytes. They have a mean of 25 different dimensions (e.g., customer hierarchy, product hierarchy, etc.) with a median of 10 dimensions.
Data Warehouse Updates
More than half of the organizations surveyed update their data warehouses on a daily basis, with only 10 percent using a trickle feed approach to achieve near real-time data.
