Enable SOA Transformation and Cross-Domain Data Fusion

Information Management Magazine, January 2008

Stephen Lahanas

It may be time for us to reconsider some of our previous assumptions regarding data integration. For decades, we’ve presumed that the best method for managing data was through strict conformance and control. We viewed the enterprise as static, remaining stable once properly defined. We’ve discovered that quite the opposite is true. Today we are facing more complexity in data integration than ever before - more data sources, greater volumes of data, more solution paradigms to deal with and greater expectations for cross-domain data exchange. Moreover, data integration has become the lynchpin within holistic architectures based upon services and sophisticated business process orchestration. Agile data architecture is not a vendor-focused solution; it is a technology-agnostic philosophy designed to address the new realities of enterprise integration. It provides a context and methodology that aligns technical solutions with the expectations that drive them. Agile data architecture provides a solution focus that pulls the big picture together without losing sight of those whom it serves, the end users.

Sometimes it is necessary to step back and reassess just what is really going on to discern the big picture. Few topics are more complex than enterprise integration. We tend to address it by breaking it up into subcategories and areas of specialization or best practices. Data architecture represents one of three primary architecture categories (data, application and process) and contains within it perhaps a dozen or so areas of specialization (warehouses, master data management [MDM], operational data stores [ODS], business intelligence [BI] and so forth). Those areas of specialization tend to be closely aligned with software solutions, but some also represent communities of practices dedicated to specific integration techniques. Each of these specializations arose to address specific business issues or integration challenges. However, what seems to be lacking is a unifying mechanism that ties all of these elements together within a relevant context. In other words, what is missing includes both the true purpose for integration and the means by which we might achieve it. Without that context, how can we hope to solve even more complex situations, such as (SOA) transformation and cross-domain data integration?

In many ways, integration is a philosophical rather than technical problem. All good philosophies begin with one central premise, a premise that is both meaningful enough to derive further elaboration and flexible enough to cover a wide spectrum of related effort. The philosophy posited in this article is dedicated to data architecture because data is the lifeblood that links all other elements of all other potential architectures together. Agile data architecture is not just philosophy, though; it also represents the beginnings of both a data and enterprise integration methodology designed to compliment rather than to combat complexity.

A Central Premise

While our central premise seems obvious upon first glance, in many ways, it isn’t. The premise is this – users are the reason why we build solutions, and users are our best resource for determining how to build them. Our focus on the user determines or ought to determine all aspects of solution design, even those which we do not generally consider to be the domain of the end user. Without a user-centric focus, technical decisions become arbitrary, and ultimately, the solution divorces itself from the reality of its conception. User-centricity is the primary motivating force behind the development of all agile solutions. The user provides:

• Immediacy – the desire for near-term real-world capability.
• Relevance and context.
• Performance expectations.
• Direction, domain knowledge and the logic behind every solution.

End users are the great democratizing force behind solution integration and development. Universal adoption is not possible without them – while we developers may be great builders and architects, sometimes what the people need are efficient and utilitarian structures rather than monuments to our own brilliance.

What is Agile Data Architecture?

Agile data architecture builds upon the core premise and represents the combination of a dynamic set of interrelated best practices rather than a standardized or single architectural approach. More importantly, it characterizes the methodology which allows that set of interrelated best practices to be coordinated and also provides us the means by which to measure our success with the resulting solution. Much of the strength behind this approach is based upon the ability of this philosophy to accommodate evolving technologies and architectural best practices. Today’s recommended solution will change, and knowing it will change will drive decisions that impact performance, cost and schedules.

Page 1 of 4.