The bright, shiny promise of a service-oriented architecture (SOA) is a happy world of applications that work together seamlessly, complete information visibility across the enterprise and provide a responsive environment that evolves quickly and easily as the IT infrastructure changes. But the reality is that even the most state-of-the-art SOA design must accommodate legacy applications, arcane but still-critical data sources and the fast-growing adoption by business users of low-cost, easy-to-adopt software as a service (SaaS) applications. Suddenly, the clear vision of rapidly implementing SOA and living in a simple, elegant world begins to cloud. The Sudoku you thought would be so straightforward to solve suddenly reveals its five-star difficulty rating - “Beware, very challenging!” All you need, though, is the right strategy to make all the answers fall into place.

The average corporation in the U.S. has dozens of disparate application systems, and some Global 2000 companies have thousands. The number and complexity of these systems will only continue to grow, and solutions for integration will have to accommodate an increasingly volatile and rapidly evolving IT infrastructure. Business processes can also undergo radical changes based on new business models, product offerings and M&As. When you factor in the explosion of connectivity needs outside the firewall (business to business, to consumers, to government), the need for extremely agile integration technology becomes obvious. A successful integration project can help organizations understand their markets and customers better, replace or upgrade applications in a nondisruptive manner, provide better business intelligence, interface with partners and add value to all existing applications by finding new ways to leverage vital systems and data.


A number of factors complicate the IT landscape:

  • Legacy systems account for a high percentage of application functionality and data storage. An integration strategy that doesn’t account for key legacy data formats only solves a small piece of the integration puzzle.
  • The myriad software systems throughout enterprises create a tangle of software asset management, version control, licensing and a host of other issues.
  • Hard-wired application links are often created as needed to join enterprise resource planning (ERP), customer relationship management (CRM), supply chain, accounting and other applications in a custom coded point-to-point style that is incredibly complex and fragile.
  • The need for B2B connections to customers and suppliers dramatically expands the number of disparate and incompatible data and application formats.
  • Companies are adopting the SaaS model for some applications, further complicating the types of endpoints that need to be included in an integration strategy.

Alongside this complicated variety of connectivity needs is the one constant in the corporate world: change. The imperatives of M&A activities that drive modern business have ensured that no single business system or IT stack will remain static for long. Each link in the integration chain can (and will) break on a periodic basis. Integration solutions that are unable to rapidly and economically adapt to this constant change are the single biggest cause of integration project failure.
The five-star Sudoku challenge is how to cost-effectively implement a process that can precisely match up with each of hundreds or even thousands of widely varied and volatile endpoints and flex with constant change without breaking.

The best strategy can be summed up in just a few key points. First, stay loose. Tightly coupled, frequently custom-coded connections in the traditional integration style mean that inevitable changes at one level require a corresponding level of change in the other layers. This rapidly increases the costs around integration and, over a period of time, dwarfs the initial license cost.

SOA offers a fresh approach and a more adaptive architecture for business integration that includes flexible technologies like Web services, asynchronous messaging, business process management (BPM) and the enterprise service bus (ESB). SOA services are self-contained, reusable, application-based units of work. They can include business functions, transactions or system service functions like show balance, check inventory, place order or receive shipment. A service requester or consumer, usually a business process, sends these requests. The service provider, any application, database, legacy system or functionality that has published its services conforming to industry standards, sends a response. In an SOA-based activity, the service request and response can be synchronous for tightly coupled, transactional needs or asynchronous so that business processes can continue independent of response. Whenever possible, go with a more robust and scalable message-passing architecture based on asynchronous, loosely coupled, coarse-grained principles. That way, if one aspect changes or fails, the rest of the system isn’t pulled down with it.

The next strategy key is an agile integration platform. Many SOA implementations today still use unwieldy and management-intensive custom code developed to solve the “last mile” integration problems. A single, unified integration middleware platform, preferably with an open metadata layer to support reusability and unify version control, gives flexibility and durability to integration connections and vastly reduces ongoing maintenance costs. Be careful of platforms that are really amalgamations of different integration applications acquired as small software companies were swallowed up by larger players. Their own internal parts can end up not integrating with each other well. Stick with a platform built on a single code base. An intuitive but not oversimplified visual design environment with a lot of power behind it is important, not just for rapid initial implementation, but for speed of repair and maintenance when the inevitable changes happen. Rapid design capability means rapid repair capability. With the sheer number of endpoints needed, the integration solution has to be executable on a variety of platforms and have a wide range of connectivity adapters, particularly to legacy data and applications. The ability to connect to SaaS applications is also essential in the modern world and is likely to become more so as time passes.

Next, solve one section of the puzzle at a time. Trying to wrap your brain around a whole page full of numbers can give anyone a headache, but solving each individual box, row or column in a Sudoku can make the goal of the whole puzzle solution attainable. Similarly, incremental adoption of any SOA integration solution, with your eye on the overall goal, can give immediate ROI while still working toward that bright, flexible integrated world. Many companies fall into the trap of huge, enterprise-wide integration projects that have an all-or-nothing “big bang” philosophy behind them. The company struggles along with the old cobbled-together, brittle solutions while the “big solution project” drags on and on, trying to hit a moving target. The business environment and IT infrastructure changes so rapidly that often by the time huge projects are finally complete, they’re obsolete.

A more pragmatic approach to use when designing integration solutions is to think of SOA as a journey or a goal to work toward. Let the line of business determine the piece of the puzzle to be tackled first. Projects that solve a particular business integration problem expose the true value of the SOA integration model: integration determined by business rather than technology factors. Service requesters and service providers can be connected ad hoc to help build out integration on an incremental basis or connected one at a time through an ESB. This doesn’t mean custom connections directly between requester processes and provider. Business processes and application and system interfaces should never be directly connected in an SOA architecture. The business process that makes a request should be completely insulated from the technology that provides the response to that request and only interact via SOA services. But if a request requires multiple applications to respond, it isn’t essential that those individual application-to-application connections be services or be connected to the ESB in the beginning.

As a place to start, determine a single request that the business process is likely to need, and plot out the flow of things that need to happen in order for that request to be fulfilled. For instance, “receive purchase order” might require that the CRM software, a SaaS application, update that a prospect has now made a purchase and is officially a customer. The accounting software, a legacy COBOL application running on a UNIX box, would need to be consulted to check for credit holds. The inventory control software, an on-premise SQL Server application, would need to be consulted to determine that the ordered items are in stock. A new service request, “fill order,” would then be triggered. An email would need to be sent to the sales representative to let him know that his prospect put in an order, for how much and how much his commission on the sale will be. This would be a single process flow unit.

Rapidly modifiable and reusable integration process flow units are very important to a robust integration architecture. They can connect either point to point or to an ESB, providing an adaptive integration layer between the high-level business process and the IT infrastructure, and can be delivered as SOA services. A good suite of integration middleware should be able to create process flows in a simple visual design environment, expose those processes as an SOA service and execute those process designs on a variety of platforms. As long as the processes support open system protocols and standards like SOAP, JCA, XML and MDB, they will fit smoothly into an SOA world. Already existing point-to-point solutions and any stop-gap measures that are in place and working well, such as SQL stored procedures, can simply be called by the process flow at first and later replaced with more robust solutions on the common middleware code base as time permits and changes demand. Not spending time fixing integration pieces that are working reasonably well keeps the bottom line low without sacrificing functionality. Among the many benefits of the process metaphor as the primary building block of integration is the ability to scale and right-size the granularity of the flows from the simplest tasks (for example, basic heterogeneous message validation and transformation) to the most complex (such as a subprocess that performs multiple coordinated data and application activities). They can even be stacked in a hierarchical manner to promote maximum reuse and abstraction for the business process layer. Significantly, all this power and flexibility will be achieved while still exposing a simple standard interface for any service requester.

An ESB as an integration backbone is an excellent long-term strategy, but it isn’t the universal panacea that many make it out to be. “Point to point” has become something of a dirty phrase these days, but a good bus system, like other aspects of the SOA model, takes time to get established with translation ports built for all the endpoints across the enterprise. Again, incremental adoption means immediate ROI and low total cost of ownership. In many situations, point-to-point connections are perfectly viable and sensible even for the long term. Whether the architecture is point to point or to a central hub, brittle, hard-wired, fine-grained connectivity is the real enemy of a flexible solution.

Change shouldn’t be a shock that shatters the system. It is a normal aspect of doing business, and the integration solution should be built at the points of contact with volatile endpoints to expect change and deal with it, not custom connected with complex code that assumes everything will always stay the same. Within process flow units, maps should be built with automatic checks and adjustments for change. A good process flow design should include extensive error checking, validation, data structure checking and an email subprocess that notifies someone immediately if human attention is needed. In case of a change, for instance, if an application is upgraded to a new version with a changed data structure, it should be a simple matter of adjusting the maps that directly touch the data structure. That’s when the value of a visual design tool that shows the difference in structure and allows quick adjustments really shines. Instead of requiring a skilled developer to alter multiple lines of obscure code to make the change, any person familiar with the integration tool and the data should be able to make the adjustment in a fraction of the time custom coding would take. With the SOA architecture in place, and multiple layers of insulation between the endpoint and the service requester, the service requester and all other aspects of the integration solution should be untouched by the change. The service requester should still be able to send a receive purchase order request, and it should still cause the proper actions to take place smoothly, with no adjustments anywhere but at the point of contact for the change.

Just as in a Sudoku puzzle, once enough processes have been built and exposed as registered services, you find a critical mass that makes the final implementation steps go far more quickly and easily than the first incremental steps. With a solid practical strategy, SOA can fully deliver on the promise of flexible, standards-based integration across the enterprise.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access