In the world of data warehousing, sometimes the only thing that appears constant is change. Convincing people to give up their fragmented reports and spreadsheets to devote time (and money) to build a data warehouse requires letting go of the old and risking the unknown of the new. The new data warehouse can cause disruption to the existing process and introduces distrust for the "new" numbers. Logic would dictate that if the new numbers are right, then someone has to have been wrong up until now. Just as soon as you deliver the first data mart or warehouse that includes everything that the business users said they wanted, they want something different. More change. How can you ever get finished with the data warehouse if people don't stop changing it? Is there no end?
The answer comes in the realization that the whole purpose of the data warehouse is to instigate change. By providing insight into the business that didn't exist before, we learn. As we learn, we can adjust and fine-tune the business to achieve our goals. The data warehouse is an instrument of change. To maximize the value of change, the data warehouse must anticipate change, adapt to it and facilitate change. It must be designed from the ground up to embrace change. How can we build in this affinity for change?
First, let's examine the things that are most likely and least likely to change in the data warehouse and how our design and approach handles change.
Things Least Likely to Change:
- Core Business Model
- Conceptual Architecture
- Modeling Principles
Things Most Likely to Change:
- Data Sources
- Number and Level of Business Users
- Level of Detail Requested
- Business Opportunities, Markets, Partners
- Sophistication of Requests
- Volume of Data
At the foundation of the data warehouse should be our understanding of the core business model and a conceptual technical architecture that accommodates change. The core business model represents a database design that has been modeled and validated by the business user. If this foundation is solid, the impact of later change will be minimal. In fact, the business users will quickly discover that even the process of articulating the business model will uncover many anomalies in the current operational systems.
Examining the rejected records from the first database load is always a surprise in that we discover that the quality of the data that is currently being used to run the business normally has some deficiencies. You can now improve quality and effect change upstream in the operational systems.
The conceptual technical architecture lays out the generic layers of the data warehouse (transformation, meta data and the user interface) to anticipate change. The transformation layer abstracts the business and cleansing rules into a reusable layer, generating the transformation logic and data output to multiple target databases. We anticipate the change of adding new data sources by reusing common cleansing rules. We accommodate the demand for larger and scalable databases by generating the target data definition language to the target database.
The meta data layer of the conceptual architecture builds on a common data dictionary and business rules that can be reused, combined and added to without changing the database. As commonly accessed data is identified, we can pre-calculate some summary data and add dimensions to our data facts to provide intuitive aggregate navigation. This allows flexibility for business users to explore and understand data without having to change (and risk corrupting) the core data model.
Using a browser-based common user interface for our conceptual architecture means that adding new views, queries and capabilities doesn't require much training time to accept change. Most business users are familiar with an Internet browser front end and can quickly adapt to added features. This also allows the implementation of a "thin" client platform that cuts costs and reduces the number of technical failure points in our business solution.
Change, in and of itself, is not bad. Indeed, it is controlled change that insures that our organization will learn, improve, compete and thrive in our business markets. The data warehouse can play an important role in stimulating change in an organization. The intelligent design of the data warehouse that anticipates change provides significant return on investment and competitive advantage for the organization.
As we lead our organization from the data darkness to the data light, there are some perks along the way. I think it was Captain Ahab in Moby Dick that said, "What I have dared, I have willed. What I have willed I will do." You (and the data warehouse) are the instrument of positive corporate change. Positive change will be recognized and rewarded. Goodbye cubicle, hello windows. Pack up the plastic plants and get ready to move to an office with a door.
Next Month: Technology Components in a Data Warehouse.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access