Data warehouses have risen from a theory scorned by academics to conventional wisdom in a short decade. Now data warehouses have become a conventional part of the infrastructure. Applications as diverse as ERP, CRM and e-commerce all rely on data warehousing as the backbone of the environment.
In corporations everywhere, data warehousing has become the norm. The type of data warehouses corporations are building can be called "formal" warehouses. A formal warehouse is one that addresses formally structured data. In a formal data warehouse, the corporation creates a data model and then sets out to find data that is patterned after the data model. Typically customer data, transaction data, supplier data, shipment data, vendor data, product data and so forth make up the contents of the formal data warehouse. The data warehouse data is integrated, granular and historical.
The foundation of data that is collected inside the formal data warehouse forms the basis for many other forms of decision support such as business intelligence, data exploration and data mining.
There is another type of data warehouse that is possible and is very different from the formal data warehouse. This type of warehouse can be called an "informal" data warehouse. An informal warehouse is a place where a very diverse, very eclectic collection of data can reside. Almost anything of corporate use can be placed in an informal warehouse. Once placed there, the informal warehouse becomes the corporate "memory" of such things, much like the formal warehouse is the corporate memory for structured data such as transactions, customers and the like.
The corporate informal warehouse might contain anything useful to the inhabitants of the corporation. Likely candidates for the informal data warehouse are articles about the company, articles about the company's industry, articles mentioning the company or its employees, internal memos, letters written to the corporation, internal corporate reports which have significant meaning, external reports generated external to the corporation but presented to the corporation which have significant meaning, e- mails, industry analysts' comments, ads placed by the corporation, ads placed by the competition, contracts, flyers circulated inside the corporation, pictures, schedules, forecasts, presentations, competitive analysis, projections and so forth.
In short, the corporation has a plethora of unstructured documents which over time tend to be forgotten. In some cases, a document deserves to be forgotten. In other cases, being able to recall and/or access a document created a long time ago can be very useful. Half of the battle is in the finding of the document. In addition, having a central facility to place and organize these miscellaneous documents is a very valuable addition a supplement to the corporate information systems.
Is it necessary to actually store these documents electronically in an informal data warehouse? The answer is that if at all possible, it is probably a good idea to store the documents in an informal data warehouse. However, given the very unstructured nature of an informal data warehouse, it may suffice to merely store a reference to the document. In other words, it may suffice to say that a contract between the corporation and another entity is in Mary Hopkins' filing cabinet. In this sense, the informal warehouse becomes a large corporate index.
The key structure of the informal warehouse might look like:
- Date of entry into the warehouse
- Document type
- Document reference number
- Physical location of document
Other useful information about the document might include:
- Document size
- Document effective date
- Document environmental factors (operating system, DBMS, release numbers)
- Disk control numbers
In short, the limitations of what the informal data warehouse ought to contain are in the heads of the designers.
One of the issues of the informal data warehouse is whether there should be links or references from the informal warehouse to the formal data warehouse. There is no reason why there should not be a link from the informal warehouse to the formal data warehouse. However, the link is not of the referential integrity variety. The link should consist of key or index information that can be manually traversed should there be a need to complete the linkage. Having a link from the formal warehouse to the informal warehouse, however, is highly unlikely.
The informal warehouse can be created and maintained using conventional technology. The only reason why conventional technology may not be useful occurs when data types are needed that are unsupported by the DBMS.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access