Traditionally, the staging area in your warehouse is pretty straightforward. A little bit of trimming and formatting, then it is off to the races on the real part of the load, right? During a recent conversation with a former client, we spoke of some considerations that went into architecting the staging area of their data warehouse project. These considerations were not technically complex, but they added significant value. We spoke of the real-world value provided by the choices we made for their application's staging area. Back before disk was so inexpensive, and before partition swapping and other data definition language commands made it fast and easy to move large amounts of data (provided there is little or no transformation), some of these things were not even considered because of the attendant cost. This column shares some of the features that have more lately added real-world value to staging exercises.
I've read arguments and had conversations with people who find value in a persistent staging area and others who think persistent staging is a waste of disk space and processing power. Those in favor talk about the ease of access to source data for reprocessing and data investigation. Those opposed generally have space concerns or small loading windows to contend with. In the latter case, anything that is not mission-critical gets left out.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access