Data warehousing teams are consistently overly focused on data and technology. Teams tend to be data driven, not process driven. This is especially damaging in two areas: user interviews and system deployment.
Data warehouse teams consistently drive the user interview process with data. Questions such as, "Can you show me all your existing reports?" and "What data would you like to see?" are the kiss of death. They instantly box you in, limit the potential range of possibilities you can deliver and ensure that the best you will ever be able to do is to automate the past. Simply delivering a new user interface on the old data will not give you a politically sustainable data warehouse system.
In user interviews, you must be process driven, not data driven. The successful interviewer concentrates on the processes of the user. They ask questions such as: "What is your mission statement? Why do you folks come to work in the morning? How do you accomplish that goal? What related processes are used to achieve your mission? How does that process relate to other functions and processes in the business?" Then they finally derive what data is required to support those processes. At the end of this interview, the interviewer has a reasonably complete understanding of where this user fits into the overall business, how they relate to other parts of the business and what functions and information are required to ensure the ongoing process. If the interviewer understands the processes related to the users, the data will always flow into that process. Process-driven designs and data warehouse systems are sustainable. Data-driven data warehouse systems tend to be inflexible, brittle systems that cannot keep up with the rapidly changing business environment.
Most technology teams spend 95 percent of their time immersed in design and build issues related to their data warehouse systems. Their idea of a test is to successfully load some data and issue a query. If the answer comes back within an hour, they declare success. They follow this with a pilot process that consists of giving a couple of users passwords and query tools and seeing if they can manage to ask a few questions and get answers. If so, full speed ahead to deployment and the inevitable parties and celebrations.
Unfortunately, data warehouse systems are primarily about process, not about technology and data. The pilot phase of the project is the place where all processes related to the data warehouse system must be developed and tested. These critical processes include both technical and user processes.
Technical processes include:
- Data process abends (extraction, transformation, load, etc.)
- Utilization monitoring
- Aggregation management
- Data replication and distribution
- Report and data set maintenance
- Meta data population and maintenance
- Access and analysis tool(s) support and maintenance
- User support
- Data quality issue resolution
- Additional data point research, development and implementation
- Additional data mart research, design and deployment
- Modification research, development and implementation
For each of these technical processes, there must be specific processes developed and tested that delineate ownership boundaries and action items to be taken to support all permutations of possible outcomes. Particular attention needs to be paid to ownership issues in the data extraction, transformation and load processes. In most organizations, this process will cross multiple functional and political boundaries. You must clearly delineate whose beeper is going to go off at all possible failure points before the occasion arises.
User processes include:
- Requesting use of the system
- Requesting additional data points
- Requesting modifications
- Reporting data quality issues
- Requesting data marts and applications
- Escalating issues to reflect chang- ing business priorities
User processes are the most often neglected and are often left to a "we'll figure it out as we go along" category. Unfortunately, in data warehousing, you never get the chance to go back and fix anything, particularly processes.
You must develop and test all processes prior to production roll out. The pilot process is your last chance to do this. Do not, under any circumstances, roll out your system until you have delineated and tested all technical and user processes associated with your data warehouse system.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access