When companies merge, no matter what the circumstances, certain economies of scale can almost always be gained by consolidating systems. Some are functional in nature, like hardware and maintenance expense, but often the larger value will be found in consolidating data assets for strategic use. Consolidating and interrogating data provides valuable insight into the successful and unsuccessful overlapping practices of both companies. This analysis can be of great benefit to identify opportunities for improvements, eliminate overlap between the organizations and provide accurate organization-wide guidance. However, to get to the value, there are architecture considerations to be addressed, tactical decisions to be made and an overall approach to be decided. So, where do you start?
The comparison of existing application functionality and requirements, all current enhancement lists, and any new organization-wide requirements should guide the scope in discussions of application consolidation. This is not a simple discussion. While many of the core business functions and metrics between the two companies may be the same, there will be incompatible hierarchies, dissimilar dimensions and business rule discrepancies. The process to drive consensus on the merged application is complex and requires a great deal of compromise. Some questions to ask include:
- What is the lowest common denominator? Where can we aggregate across organizations and truly compare apples to apples?
- Does it make sense to combine certain applications, or maybe just parts of the applications, or is it better to let them continue to operate separately?
- What is the tie-breaking process for tough decisions? When different groups want different things, who makes the final call and how do you make it happen quickly?
Once all of the important data sources have been identified and the rules about processing and conforming the data from the different organizations are decided, we can move on to architecture. Several architecture options to consider are specific to the migration effort. One approach to consider is use of data from the transaction systems, rather than the existing analytic applications for backload. Often, if this approach is available, the same programs that consolidate the data can be used in the ongoing processes. This helps to provide consistency in business rule application.
Remember when writing a one-time load conversion process, the fact that it appears to only be required to run once is no reason to skimp on design effort, testing and tuning. Even if you can’t use the transaction system to do the backload, there is no way the process will really only run once. During the testing process, it will be run many times, so the code needs to be solid and follow standards. The code used for migration may be referred to in the future if there are questions about how business rules were applied during migration. Performance is important – pay attention to it. There’s nothing like coming down to the wire, planning for go-live and realizing that the historic load will take six months when the timeline allowed two weeks. It doesn’t have to be that way. If data volumes are high, special attention is required. It isn’t hard to rack up long load times for systems that load data constantly. It starts with something like, “It is going to take 24 hours to load a week’s worth of detail data to the warehouse and then to the marts that it feeds. We’re loading five years of historic data.” Well, math tells me that there are 260 weeks in five years. Left untuned, that translates to 260 days of historic loading – that’s eight months. Planning for performance-tuning time makes all the difference. Don’t underestimate the amount of time required to design and tune the historic load solution. On some projects, I’ve made a spot for a dedicated resource to handle these tasks. For the time period that your new consolidated reporting application might be receiving feeds from multiple legacy transaction systems, consider a load status Web page. The page should include all relevant source systems and the date to which data has been loaded. This helps avoid confusion about whether the application is up to date.
While data migration can mean many different things, some rules can be applied to all situations regarding the need to apply consistent processes, migration validation and understanding that a one-time migration will be run many times. When attempting to integrate the data assets of merging companies, focusing on the overall strategic benefits and a few tactical maneuvers can help your project be a success.
Don Steffen is a cofounder and partner of AmberLeaf Partners, Inc. (formerly BI Solutions, Inc.), a consulting firm dedicated to enabling innovative companies with the information to make critical investment decisions. Steffen has been designing and delivering technical architecture and solutions in the business intelligence and data warehouse industry for more than a decade. He can be reached at dsteffen@amberleaf.net.
Be the first to comment on this post using the section below.