The adage of “No Pain, No Gain” isn’t just for sports. While all organizations fantasize about a fully-integrated IT paradise that works seamlessly across departments, groups and geographies, few actually sign up to experience the pain involved in the process of integration.

The only reason anybody spends money on integration at all is because software, as a rule, doesn't integrate by itself. But no executive thinks that spending money on integration addresses a strategic need of the business. Instead, money spent on integration typically goes into fixing something that really shouldn't have been broken in the first place.

Based on the traditional approach, a software application is defined with system-based requirements, its design, and then coding and testing within a well-defined software development lifecycle. The primary focus is on the code's implementation. As a result, integration with teams has traditionally been tightly coupled with little consideration for metadata. Today, integration solutions and paradigms have shifted the focus to code re-use and consolidated application solutions. This approach redefines the solution based on business-defined services - which is how it should have been done from the start.

With the introduction of service-oriented architecture, the paradigm of business users creating application functionality was very exciting and led to a lot of buzz. This was done through building and managing business processes, all hosted on an enterprise service bus - thus disentangling the integration “spaghetti” forever. SOA abstracts the underlying technology, so that developers enmeshed in connecting systems and applications can now focus on building services. With a selling proposition like that, it was no surprise that everyone wanted a piece of the action.

What Isn't Working with SOA

However, for all the excitement SOA generated, the enthusiasm has dimmed a bit somewhere down the line. So, what happened?

SOA is usually mistaken for a set of Web services defining a complete solution. Rather, SOA is a technology paradigm that supports the processes defined and governed by business. There has been a considerable investment in building SOA infrastructure already, but the payback has been slow, leading companies to invest in development of Web services rather than define a pure service bus.

There is a perception of 'SOA' as a means to enable integration with legacy systems. Legacy encapsulation can certainly be a great enabler for business processes, but it doesn't substitute for an effective SOA strategy.

A successful SOA initiative should focus on reducing complexity, transforming systems into services that implement core capabilities and eliminating redundancy. Further, it must define common patterns that apply to all business units and set up 'software factories' that enable the delivery of new products as services in a fraction of the time it used to take.

A successful SOA initiative must begin with strategic investment and aim at lowering total cost of ownership as subsequent implementations takes place. For an organization to adopt this strategy, it needs to be mature enough to take the risk of initial investment. Organizations with a clear focus on implementation will reap long-term benefits. To define the SOA roadmap well, it is critical to understand that it is about processes or services definition as well as technology and people. (See Figure 1, left.)

What Should We Be Thinking About?

Cut through the hype, be realistic. SOA has been overhyped, oversold and set up to fail. Headlines decrying the failure of strategic SOA projects tell tales of woe. In many cases, organizations that merely needed some straightforward extensions to their existing architecture were up-sold unwieldy architectures that proved impossible to deploy. It's a bit like starting to build a bypass for a town, but somewhere along the way the project grew to the size of a major city. Some useful reality checks:

  • Ensure relevance by grading usefulness of each solution fit.
  • Start small and deliver business value before becoming more ambitious.
  • Define a roadmap for the entire service lifecycle, not just service identification.
  • Ensure that SOA is requirements-driven and not demand-driven.

Crack the whip and rein them in. Governance is one of the most abused terms in the SOA world. Many on the SOA team manage architectural principle exceptions to orchestrate grossly inappropriate solution architectures. Those who question them are labeled prudes and dismissed in the name of progressive architecture. It is imperative to determine and price-out the SOA solutions that actually provide significant value to business. Applications that provide very little value to business but cost an awful lot are the ones to eliminate first. Doing SOA right requires core services and infrastructure to be built before the high-visibility, high-value projects can be properly accomplished. It is important to ensure strict adherence of key projects to all governance mandates.

Skip the canonical data model. Purists may shake their heads, but our experience has been that the enterprise canonical data model is not indispensable for an SOA initiative. The way forward is a federated MDM strategy and a brokerage approach to communication between systems. This should ideally be based around a distributed data strategy that leaves information in the source systems but provides references between them. If you consider the impossibility of imposing the canonical data model on your next SaaS provider, the virtue of this approach becomes apparent.

Get on the dirty bus. The real enterprise service bus is a myth. The entire enterprise will never get on a single “bus.” This is primarily because there never will be a business case strong enough pull funding for the strategic service model on legacy systems. A more pragmatic approach would be to divide the enterprise bus into two logical sections: the clean data bus that hosts enterprise services adhering to the enterprise service framework, and the dirty bus to host specific services for legacy and non-adhering systems. The dirty bus clearly scores over the erstwhile point-to-point integration, offering greater control over auditing, logging and exception handling.

Socialize and SLA(y) them. In order to ensure that SOA is useful, it will need to be used. Many an organization has lost the fruits of the SOA karma earned over the years because the services developed were not socialized enough. It is essential that the services are published through an elaborate (yet simple to read) service catalog to ensure that they are available across the enterprise for use. The other crucial factor in ensuring reuse (and indeed use as well) is to make results predictable. This is enforced through well-defined and closely monitored service level agreements. SLAs also bolster the business case by ensuring higher level of reliability in meeting business objectives.

So(A), Where Should We Be Looking To Go?

(See related Figure 2, left.)
Is the future cloudy? Cloud computing is the rage today. It is obvious that SOA and cloud computing go hand-in-hand. Cloud computing encompasses any subscription-based or pay-per-use service that, in real time over the Internet, extends IT's existing capabilities. In essence, this is distributed computing. An application is built using the resource from multiple services, potentially from multiple locations. Behind the service interface is usually a grid of computers to provide the resources. The grid is typically hosted by one company and consists of a homogeneous environment of hardware and software, making it easier to support and maintain. Nothing really changes outside of that, including the need to do SOA properly.

Information as a service. Information as a service accepts the idea that data resides within many systems and repositories. The new paradigm's trick is to enable standardized access to this disparate data. Developing vast databases that have been wrapped and delivered as business intelligence snippets to the whole enterprise vastly increases the reach of critical BI. When you combine this concept with the federated mantra of cloud computing and build it on top of an agile SOA-based platform, the nebulous vision becomes much clearer.

Integration as a service. Right now, three critical technologies (connectivity, policy enforcement and semantics) are available as capabilities and/or services within the SOA. The future will see IaaS infrastructure embedded in hardware or offered more often as managed services. Capabilities, such as analytics, will be offered as a real-time service. And this isn’t too far in the future: A major telecommunications company is now manufacturing an appliance with SOA integration built in. Attached in your basement, it connects you to services provisioned from within the company's network. These services go way beyond mere routing and transformation and provide full-blooded process orchestration capabilities.

Lean and agile competency centers. In this rapidly changing business and technology landscape, the first-generation integration competency centers struggled to constantly deliver significant efficiencies, making it imperative to go beyond the obvious. Most organizations are moving toward defining a framework through the transformation of their organization into a “Lean ICC” - one that is agile, efficient and adaptable in responding to opportunities and threats. The Lean ICC shortens the awareness-to-adoption cycle with an accelerated ICC deployment process accompanied by comprehensive but crisp communication and change management. With a highly modularized yet integrated framework, such competency centers are tailored to the specific needs of business, with prioritization based on market dynamics, enterprise inertia, business priorities, investment strategy and ROI.

To get ahead in the game, organizations must align IT with business strategy. Leveraging existing systems, ensuring acceptance across the enterprise and readily adapting to change can improve performance for competitive advantage. SOA is one critical change agent that can no longer be excluded from any serious IT strategy that aims to be a growth-enabler. Architectural assets without the overhead of owning, operating and maintaining them makes eminent sense.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access