DM Review welcomes Ralph Kimball as a new columnist. The Kimball Perspectives series will systematically describe classic best practices as well as new trends in technologies. The most important first step in designing a data warehouse (DW)/business intelligence (BI) system, paradoxically, is to stop. Step back for a week, and be absolutely sure you have a sufficiently broad perspective on all the requirements that surround your project. The DW/BI design task is a daunting intellectual challenge, and it is not easy to step far enough back from the problem to protect yourself from embarrassing or career-threatening changes discovered after the project is underway. Before cutting any code, designing any tables or making a major hardware or software purchase, take a week to write down thoughtful, high-quality answers to the following 10 questions, each of which is a reality that will come to control your project at some point. These define the classic set of simultaneous constraints faced by every DW/BI effort.
Time spent answering these classic DW questions is enormously valuable. Every one of the answers will affect the architecture, choice of approaches and even the feasibility of your DW/BI project. You dare not start coding before all the team members understand what these answers mean! The big news is that end users have seized control of the DW. They may not be building the technical infrastructure, but they are quite sure that they own the data warehouse and the BI tools and that they must meet their needs. This transfer of initiative from IT to the end users has been very obvious in the last two or three years. Witness the soul-searching articles and industry speeches exhorting CIOs to show more business leadership and the high CIO turnover as reported in CIO Magazine. Many of the 10 questions in this column are brought into much clearer focus by increased end-user ownership of the DW/BI system. Lets focus on the top five new urgent topics, in some cases coalescing our questions: Business requirements.The DW/BI system needs a permanent KPI team continuously in touch with end-users analytic needs and the consequent demand for new data sources to support new KPIs. Also, the system should increasingly support the full gamut of analytic applications, which include not only data delivery, but alerting the users to problems and opportunities, exploring causal factors with additional data sources, testing what-if scenarios to evaluate possible decisions and tracking the decisions made. The DW/BI system is not just about displaying reports, but rather must be a platform for decision-making in the broadest sense. The oldest label for data warehousing, decision support, remains surprisingly apt. Strategic data profiling. The earlier you tell the users bad news about the viability of a proposed data source, the more they will appreciate you. Develop the ability to assess a data source within a day or two. Elevate the data profiling tool to a strategic, must-have status. Tactical data profiling. The increased awareness of data quality is one of the most remarkable new DW perspectives, certainly driven by end users. But all is for naught if the business is not willing to support a quality culture and the end-to-end business process re-engineering required. Integration and latency. The end-user demand for the 360-degree integrated view of the business has been more like an approaching express train than a shock wave. We have been talking about it for almost a decade. But now the demands of integration coupled with real-time access to information have combined these two issues into a significant new architectural challenge. Compliance and security. DW/BI folks in IT often dont have the right instincts for protecting data, since the system is supposed to be about exposing data. But this new emphasis on compliance and security can be built systematically into the data flows and the end-user tools across the entire DW/BI system. The purpose of this first column has been to expose the fundamental design issues every DW/BI design team faces and to bring to the surface the urgent new requirements. In this ongoing series of columns, I will have the chance to probe each of these areas in some depth, reminding us of the remarkably unchanging aspects of data warehousing, while at the same time trying to catch the winds of change.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access