Question: We don’t seem to be able to get it right. When we deliver a data warehouse solution, the users tell us it doesn’t match what they need. We went through what we thought were the right steps to gather the business requirements, but either the requirements have changed or we never really understood what they wanted. Help!


Chuck Kelley’s Answer:How long did it take before you delivered the data?My guess is that it was longer than three months. Within three months, the requirements generally change and since you did not ask during the development if the requirements changed, you were not notified.


I suggest putting together a team of folks (business and technical) and a development process that take the business requirements (small number of them) and product something just about every month to make sure that you are on track.If they keep saying "yes," then they won’t say no at the end.


Larissa Moss’s Answer: I strongly suggest an agile approach. I call it "extreme scoping." The first thing to do is to break the habit of delivering an application with one project. While most organizations realize, by now, that they cannot build an entire data warehouse (DW) all at once, they don’t realize that they cannot and should not build an application all at once either.


Why? Some answers are:


  • The requirements are most likely unstable;
  • The scope is probably too large or the deadline is too tight;
  • Some technology components might be unproven;
  • Data integration complexity is most likely not fully understood;
  • The quality of the source data probably has not been vigorously profiled because many (if not most) business rules are not documented or even known;
  • The project team may be too small, too large or untrained; and
  • Business users are probably too busy to provide continuous feedback.

With a project like this, following a traditional methodology and staying on paper for weeks or months is the kiss of death. You have to get physical as quickly as possible. That means prototyping, prototyping and more prototyping.


How do you prototype an entire application without falling back into the waterfall sequence of requirements, analysis, design, coding and testing? You break the application into multiple software releases. The scope of each software release will contain only a small portion of the application requirements, hence the name extreme scoping. Each software release becomes a project. Each project is developed using an iteractive prototyping approach. Each prototype is time-boxed (anywhere from 10 to 60 days). Most software releases should produce a production-worthy deliverable (although, there are occasions where you want to further refine refactor, a deliverable in the next software release, before putting it into production). After several software releases, you will have a completed and fully-functioning application.

The quality of the application will be higher with this approach than with a traditional waterfall approach, because mistakes can be found and fixed early in the development process. With the extreme scoping approach, you have sliced the development process vertically across the application scope instead of horizontally across the development phases. This allows:

  • The requirements to be refined with each software release until they become stable;
  • The scope of each software release to be small enough to fit into a tight deadline;
  • Technology components to be evaluated early on;
  • Data integration complexity to be handled in small increments;
  • The quality of the source data to be vigorously profiled, because you will incrementally find and document the business rules;
  • The project team to be small or even in training during the early software releases with minimal impact on the entire application;
  • Business users to be more involved in prototyping activities than on traditional projects where they hand over requirements and wait for user acceptance testing; and
  • Many mistakes or misunderstandings, that cannot be found on paper and only found once the entire application is in production, to be found and corrected during the development process, before the entire application is completed.

For more information, read my article "Extreme Scoping" in the Library EIMI Archives or attend one of my seminars on agile project management occasionally offered publicly at TDWI conferences.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access