Four Golden Rules of Successful Data Migration

Not all migrations are equal. Many migrations are fairly straightforward and motivated purely by technical or IT drivers. For example, at the storage level companies might need to migrate their data to cheaper forms of storage, and at the database level they might need to mirror data to another disk. At the application level, however, data migration can be very complex in large enterprises, because it needs to span both technical and business issues. According to Bloor’s Phil Howard, Forbes 2,000 companies already spend at least $5 billion per year on migrations, yet 80 percent of them still go over time or over budget.¹

To understand why this is the case, it is important to recognize how complex IT infrastructures have arisen. Often, they mirror an enterprise’s history and have come about as the result of mergers and aquisitions (M&As), organic growth, product launches and new initiatives over many years. While individual parts of the infrastructure might be well architected, robust and functional, the overall structure might be poorly understood and far from optimal. However, while IT optimization might be desirable, unpicking this complexity is far from easy and risks catastrophic failure, such as lack of availability of business-critical data or failure of service or product delivery systems. However, analyzing successful IT migration projects reveals common success factors, as shown in Figure 1.

Rule one: Data migration is a business, not a technical, issue. Historically, migration has been viewed as a purely technical issue, and the most pressing concern has been how to migrate data. But because IT doesn’t necessarily own either the source/target applications or the data that the business uses to function, it doesn’t have the power or the necessary knowledge to deliver what is required by the business. Further, an increasing feature of the modern IT environment is that IT has come out of the data center and is no longer controlled by the IT department in the way that it might have been previously. Involving business managers in data migration projects is therefore essential. In fact, one of the most common characteristics of successful migrations is they are business led rather than technology led. Putting the business in the driver’s seat means that before we ask, “How do we migrate data?” we first answer a series of important related questions that help us frame and scope the project:

• Why are we migrating data
• What data should be migrated
• When should it be migrated?

These questions cannot be answered by technicians. They can only be answered by business managers, which is one reason why it is essential for the business to drive the migration. Ensuring that the business makes the decisions and drives the project also frees IT to do what it is best at, which is handling the technical aspects of moving the data.

This suggests any technical solution to the migration must provide the business with clear visibility and control over the way the program is progressing.

Rule two: The business knows best. The second rule of successful data migration is closely linked to the first. The business drivers, not technical ones, should take precedence. It is critically important that business goals define the solution and approach selected, and not the other way around. The best technical solution is not always the best business solution. Therefore, to be successful, the chief business stakeholders must define their requirements and take responsibility for driving the project.

The business is also better placed to prioritize the migration, deciding which data to migrate first. For example, it might decide that migration be triggered by a customer selecting a new service that can only be supported on the new application stack. Or it might decide that the most important customers be migrated first, or second, or last according to business need. By allowing the business to drive the migration, an enterprise has a better chance of delivering the right solution for its needs and maximizing ROI.

However, IT must be aware that the business will not be able to predict at the outset what issues will surface during the migration. Moreover, in a long-running program they will absolutely have to handle changes in priority and direction. The chosen migration technology must be able to support such changes without restarting every time.

Rule three: No one needs, wants or will pay for perfect data. Applications are only as good as the data they have available to them. We also know that many data migrations have been scuppered by overestimating or not understanding the quality of the source data. Oh, the joy of legacy data with its gaps, inconsistencies and redundancies! While enhancing data quality is a worthy goal, it is really important not to go off on a tangent midproject in the quest for perfect data quality. Like overspecification of an application, the quest for data perfection can result in negative consequences for the project. It is where many, many projects run aground - inflating both the cost and the time to deliver the project. To avoid this trap, data owners and users need to determine the level of quality they require at the start of the project, in order that the technologists have an appropriate goal. Project managers must also be aware of the quality of their legacy data and allow adequate time and budget to achieve the requisite data quality.