Data migration can mean different things given a certain situation or task. Migration can be necessary to seed a transactional system, to move data to facilitate a hardware upgrade, to convert a legacy reporting system to a new data warehouse and marts, or to provide a consolidated view of a merged company’s data assets. Establishing a data migration strategy can be complicated, but there are some tips and tricks to maximize data assets when companies join forces. Let’s focus specifically on reporting and analytic applications.
When companies merge, no matter what the circumstances, certain economies of scale can almost always be gained by consolidating systems. Some are functional in nature, like hardware and maintenance expense, but often the larger value will be found in consolidating data assets for strategic use. Consolidating and interrogating data provides valuable insight into the successful and unsuccessful overlapping practices of both companies. This analysis can be of great benefit to identify opportunities for improvements, eliminate overlap between the organizations and provide accurate organization-wide guidance. However, to get to the value, there are architecture considerations to be addressed, tactical decisions to be made and an overall approach to be decided. So, where do you start?
The comparison of existing application functionality and requirements, all current enhancement lists, and any new organization-wide requirements should guide the scope in discussions of application consolidation. This is not a simple discussion. While many of the core business functions and metrics between the two companies may be the same, there will be incompatible hierarchies, dissimilar dimensions and business rule discrepancies. The process to drive consensus on the merged application is complex and requires a great deal of compromise. Some questions to ask include:
- What is the lowest common denominator? Where can we aggregate across organizations and truly compare apples to apples?
- Does it make sense to combine certain applications, or maybe just parts of the applications, or is it better to let them continue to operate separately?
- What is the tie-breaking process for tough decisions? When different groups want different things, who makes the final call and how do you make it happen quickly?
Once all of the important data sources have been identified and the rules about processing and conforming the data from the different organizations are decided, we can move on to architecture. Several architecture options to consider are specific to the migration effort. One approach to consider is use of data from the transaction systems, rather than the existing analytic applications for backload. Often, if this approach is available, the same programs that consolidate the data can be used in the ongoing processes. This helps to provide consistency in business rule application.
Remember when writing a one-time load conversion process, the fact that it appears to only be required to run once is no reason to skimp on design effort, testing and tuning. Even if you can’t use the transaction system to do the backload, there is no way the process will really only run once. During the testing process, it will be run many times, so the code needs to be solid and follow standards. The code used for migration may be referred to in the future if there are questions about how business rules were applied during migration. Performance is important – pay attention to it. There’s nothing like coming down to the wire, planning for go-live and realizing that the historic load will take six months when the timeline allowed two weeks. It doesn’t have to be that way. If data volumes are high, special attention is required. It isn’t hard to rack up long load times for systems that load data constantly. It starts with something like, “It is going to take 24 hours to load a week’s worth of detail data to the warehouse and then to the marts that it feeds. We’re loading five years of historic data.” Well, math tells me that there are 260 weeks in five years. Left untuned, that translates to 260 days of historic loading – that’s eight months. Planning for performance-tuning time makes all the difference. Don’t underestimate the amount of time required to design and tune the historic load solution. On some projects, I’ve made a spot for a dedicated resource to handle these tasks. For the time period that your new consolidated reporting application might be receiving feeds from multiple legacy transaction systems, consider a load status Web page. The page should include all relevant source systems and the date to which data has been loaded. This helps avoid confusion about whether the application is up to date.
While data migration can mean many different things, some rules can be applied to all situations regarding the need to apply consistent processes, migration validation and understanding that a one-time migration will be run many times. When attempting to integrate the data assets of merging companies, focusing on the overall strategic benefits and a few tactical maneuvers can help your project be a success.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access