Is Agile Enterprise Data Modeling an Oxymoron?
So you have heard of the oxymoron jumbo shrimp, but have you Heard of agileenterprise data modeling? In a training class recently, we were discussing how to build an enterprise data model, and I used the phrase "agile enterprise data modeling." Is it possible? Can you build an enterprise data model using an agile approach? Why or why not?
We can broadly define "agile" as a collaborative and iterative approach for producing application deliverables one small piece at a time within very tight time frames. The collaboration and iteration focus of agile methods can be well leveraged when building an enterprise data model. A majority of projects driven by agility, however, lack a big-picture focus and often strive to deliver small slices of functionality within tight time frames, at times redoing and revamping prior work. If we can initially create at least a high-level, broad enterprise view or reference an existing industry data model, than we can confidently deliver small pieces of functionality knowing each piece fits with other pieces to create a well-connected enterprise data model. Let's see how iteration and collaboration can make an agile approach possible for building an enterprise data model, provided each small piece delivered fits together well with all of the previously delivered pieces.
Agile methods promote iteration, and an enterprise data model must be produced iteratively. Huroye Scott, senior data architect, Bruce Weidenhamer, data architect, and Johnny Gay, data analyst/architect, all emphasize the need to always think iteratively and incrementally when it comes to enterprise data modeling. Johnny says, "We stop when it's good enough - taking baby steps toward perfection, one project at a time. We've all seen the countless failed attempts to create a useable enterprise data model with one extremely long project, which invariably ends with an obsolete picture long before the ink dries." Huroye adds, "An enterprise data architecture is always changing, and that calls for an approach that is adaptive and iterative."
Agile methods promote collaboration, and an enterprise data model must be produced collaboratively. Data architect Art Trifonov says, "One aspect of Agile that I find useful in my practice is the emphasis on interpersonal communication over formal documentation. Formal documents and intermediaries cannot always be relied upon to uncover and explain fine subtleties of relationships." Richard Leach, data specialist, adds, "Many have colocated teams, all have some form of daily update process and a chance to share problems and successes. While this may not be quite so obvious with data models, the end result is far more likely to be accepted and understood in whatever form it arises."
Agile methods promote frequent delivery of small pieces of the project, and each piece of the enterprise model produced must fit together with all previous pieces produced. Data warehouse architect Maria Gonzalez recommends starting with a high-level enterprise data model. Maria says, "An enterprise data model first iteration needs to be that high-level global overview, so that when you start running iterations with more specific/limited scope, you don't end up realizing that you need to redo your previous iterations." Enterprise data architect Jarrod Luchsinger, data architect Karen Forcht and IT architect Scott Meredith suggest an industry data model might be useful as a starting point. Karen says, "I believe that if you have a blueprint of your enterprise model already defined, you can build the details of your enterprise data model in an agile fashion. We are doing that today. We have purchased an industry standard model as our framework to reference, but we are building out the details subject area by subject area as projects come up." Scott adds, "Each agile method iteration is supposed to provide a usable product to the end user. I find it difficult to imagine being able to get the view of an overall enterprise in such a short iteration. However, if an industry model is used as a framework, I think an agile methodology has potential. As long as the overall relationships between areas of the model are defined, you could use an agile iteration to refine parts of the enterprise model."
Marcin Kulakowski, data engineer, summarizes, "The agile methodology is a great way to execute a project. It has many advantages and has been proven to get software products to market faster with fewer defects than traditional methods. Despite the advantages, it is not a good fit for all projects and care should be taken to apply agile techniques in the right places."
I believe if care is taken, then yes, we can build an enterprise data model using an agile approach leveraging iteration and collaboration. We must initially craft a high-level enterprise view as context for each slice of the enterprise data model we complete.
Join in the design challenge by becoming a Design Challenger. Sign up at stevehoberman.com.