At Baseline Consulting we’ve been talking for several years about the concept of a data supply chain. But IT executives are only now starting to catch on to its importance.
Over the past 15 years there has been a big push to standardize on off-the-shelf software. This allowed IT organizations to buy instead of build. We’ve migrated from proprietary architectures to Windows and Linux standards. We’ve gone from custom-built applications to packaged CRM and ERP applications. IT adopted this approach because its value is automating business processes and supporting analysis– not inventing new technologies. The problem is that moving data between all of these “packaged systems” still requires custom code.
There’s no question that middleware provides value: it delivers the pre-built data pipes. Unfortunately, these are toolkits requiring developers to write code to connect their packages to the pipes. Most CIOs are blissfully unaware of the amount of custom coding middleware requires. Trust me: IT spends an enormous amount of money on supporting such data migration solutions. Many IT shops still view middleware as sacred ground.
The data warehousing world has enthusiastically adopted ETL tools to reduce custom coding so they can focus on the issues of data accuracy and usability. One fact lost in translation is that ETL integrates data – it’s more than just a pipe. The application world has adopted EAI, ESB, and orchestration to move data quicker. However, there’s no integration. Each application is responsible for integrating the data they receive.
Custom code to move data around isn’t the answer. Orchestration, message passing, and data movement just creates a labyrinth of pipes. There are no economies of scale. The data doesn’t get better.
Walmart learned years ago that it was impractical to have a custom (and separate) distribution system for every supplier. They knew the cost benefits of a standard distribution system; this meant they needed to standardize the size of the trailers, the size of the boxes, and the way the boxes were packed and shipped. The benefits of a supply chain is that standardization occurs at the most cost effective point: the source. Walmart’s distribution success was measured by its ability to accept new suppliers and manage more shipments.
Most CIOs don’t recognize that they have a data supply chain. Instead of building a custom distribution system for each suppler (each business application), they should be focused on a single data supply chain. Middleware supports the creation of custom distribution solutions, but not the standardization of data. A data supply chain can only be successful if the data is standardized. Otherwise everyone is forced to write custom code to standardize, clean, and integrate the data.
Evan Levy's blog can also be found at http://baseline-consulting.typepad.com/evanlevy/.
A well written explanation of the "dark science" of integration. Even those tools built to simplify this process are themselves complicating the process. Recognizing the crippling Integration Risk that all enterprise systems experience we have created an industry first: Enterprise Add-On.
This application, patent-pending, is predicated upon the fact that all of the Enterprise Applications are good at what they do: consolidate operations and standardize transactions. Where we come in is what they can't do and solves the integration risk issue. Enterprise applications, including ERP, Best of Breed, HIS, etc. are rigid and unforgiving when it comes to adopting change. Operations is on the outside wanting to change and IT is in the unenviable position of either saying No or putting the company at risk, financial and time, to making the change.
Babbleware developed its application to pick up where these Enterprsie Applications leave off by using their "interfaces" and not integration, but literally the "broadcast" signal they use today. These include Telnet, XML, XLS etc. We recognized that while the format of each of these "sources" was unique, the structure they contained was repeated, therefore able to be learned, and then enhanced and extended into the change of technology, data or process business operations requires without any change to existing systems. In fact, the existing system is blissfully unaware Babbleware's Enterprise Add-0n even exists.
Our early success has indicated the customers appreciate the opportunity to protect their existing investments, avoid on-going development/maintenance/upgrade costs associated with touching their enterprise applications, and unleash innovation in their business without the constraints they have traditionally faced. There is hope to avoid the problem you so clearly identified.