The other day I was in London and I heard a friend complain about the cost of getting from the airport to downtown London. My friend said it cost him £3,500 to get from Heathrow to Kensington. Having just taken the tube for two pounds I could not imagine what his problem was. How in the world did it cost you £3,500?" I asked. My friend then commenced to say that for the trip he had rented a horsedrawn carriage with a driver dressed out in a white wig and a 17th century jacket brocaded in gold. The attendant in the back of the carriage wore similar clothes and blew a long brass trumpet, similar to one that might be found in Pimlico or Epsom Downs. The horses pulling the carriage were Arabian stallions, and the carriage had last been used in the wedding of Charles and Di. The horses were adorned with brass and leather and pulled the carriage using fittings made in the 16th century. The trip was the talk of London and even made the French and American newspapers. And still my friend complained about the cost of it all.
If you want to take a trip and are worried about expenses, you do not buy every expensive and ornamental adornment known to man.
Data warehousing is the same way. People grumble about the cost of their warehouse, having built and supported a structure that can only be described as Byzantine. When it comes to warehouses, there is an amazing amount of money that is simply being thrown away that corporations either don't know about or care about. Corporations do not care or bother to build a warehouse properly. They are in such a hurry that they do not understand that there is a right way to build a warehouse and a wrong way, that there is a proper structuring of a warehouse and an improper structuring. However, after they have created their serpentine labyrinths, corporations will complain long and hard about the outrageous expenses of their monstrosities. It is another irony--which defies rationality--that corporations willfully waste huge amounts of money and then complain about expenses.
The biggest factor in the managing of the cost of construction and operation of the warehouse is not the cost of the technology that houses the warehouse, but the skill and understanding of the designer and the data warehouse administrator.
Before you start to complain about your warehouse costs, take the following test to see how cost-effective your warehouse is.
To begin the test, start your point count at zero. And remember, this is a self- administered test; so if you cheat, you are only fooling yourself.
- Add 20 points if you are using a tool of automation for the creation and maintenance of the code required to transform and move legacy data into data warehouse data. Add no points if you are manually creating and maintaining the code necessary for integration, transformation and movement. Subtract 5 points if you think that replication is how data gets from the legacy environment into the data warehouse environment.
- Add 30 points if your warehouse resides on a combination of disk storage and near-line storage. Add no points if your warehouse resides on disk storage exclusively.
- Add 10 points if your monthly usage ratio is greater than .5. Add 5 points if your monthly usage ratio is greater than .2. Subtract 5 points if you don't know what a monthly usage ratio is. Subtract another 5 points if you don't know how to calculate a monthly usage ratio.
- Add 5 points if you regularly monitor the usage of your warehouse. Add 1 point each if you can identify this month's top ten users, queries, moments of peak utilization, most accessed tables and most accessed columns.
- Add 25 points if your warehouse was built iteratively under a spiral methodology. Add no points if your warehouse was built with a classical waterfall methodology. Subtract 5 points if you do not know what a spiral development methodology is. Add 5 points if your warehouse continues to be built with a spiral methodology. Subtract 20 points if you built your data warehouse under the "big bang" philosophy.
- Add 10 points if log or journal tapes are used as a source of ongoing refreshment. Add no points if you have to scan a database in the legacy environment in order to determine what data should be refreshed into the data warehouse. Subtract 10 points if you do periodic wholesale refreshment of data into the data warehouse.
- Subtract 5 points if you regularly do update into the data warehouse during peak processing hours.
- Add 5 points if you regularly purge data from your data warehouse. Add another 5 points if the data that has been purged is purged on the basis of a known low probability of access.
- Add 20 points if you built your enterprise data warehouse before you built your data marts. Subtract 5 points for every data mart you built before you discovered that you need an enterprise data warehouse in order to properly build and source a data mart.
- If you use the services of a consultant, add 15 points if the consultant considers his/her first task to be making you independent of the consultant. Subtract 10 points if your consultant thinks he/she is there to deliver a data warehouse but not to transfer skills. Subtract another 5 points if your consultant has sold you on doing the development on the basis of using "bait-and-switch" tactics.
- Add 5 points if you manage your consultant in terms of deliverables that are achieved in no more than six months. Subtract 10 points if you have a contract with a consultant that is longer than a year.
- Add 10 points if you used a data model for the basis of design for your warehouse. Add another 5 points if your data model had already been developed in house or if you acquired the data model from an external source. Subtract 10 points if you had to manually develop the data model for the express purpose of building the data warehouse.
- Add 10 points if you included the end user heavily in the early stages of the building of the data warehouse.
- Add 5 points if you did capacity analysis at the outset of the development project. Add 10 more points if your capacity estimate was an order of magnitude within your actual usage amount.
- Add 1 point if meta data is a regular part of your infrastructure. Add 10 points if you are implementing distributed meta data.
- Add 5 points if you considered your hardware vendor's track record with data warehousing before making a selection. Subtract 5 points if your hardware vendor does not know the difference between operational systems and informational systems.
- Add 10 points if you considered your DBMS vendor's track record before purchasing your DBMS. Subtract 10 points if your DBMS vendor still thinks that a database serves all purposes and users at the same time.
- Subtract 5 points if the first activity of data warehousing that your company did was to select an end-user access and analysis tool.
- Add 10 points if your consultant or designer can define and diagram the architecture showing how all of the following fit:
- an enterprise data warehouse
- a data mart
- an operational data store
- an exploration warehouse
- distributed meta data
- the layer of integration and transfor- mation programs
How did you do? Here's a little scale to rate yourself.
230 - 180 points-- Congratula-tions. You have done a great job. But, if you are worried about data warehouse costs, you are entitled to scream. In fact, give me a call and I will help you.
180 - 100 points--You have done a good job of controlling costs, but there are a few improvements to be made. You are entitled to a good loud shout, but do not call me.
100 - 50 points--Why are you complaining? You built your warehouse in a shoddy manner, so who do you have to blame? Yourself. You are entitled to a whimper, but don't let many people hear you, especially me.
50 - 0 points--You bought a pig in a poke. You should have been a lot smarter before you set out to build a warehouse. You are entitled to a tiny peep, but you have to promise to do it in a closet.
Less than 0--Why are you in a position of budget authority? You obviously don't know the first thing about warehousing, only spending money. You are entitled to nothing; but if you do decide to whimper, do so softly, in a closet, with the lights off, covering your mouth with a pillow.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access