On the list of things guaranteed to give you a migraine, data migration is a close second to fingernails on a chalkboard. Even the best-laid plans always seem to have something go awry. Whether it is the unexpected data source discovered in the final hour, the laundry list of issues from user testing or the seemingly endless budget overruns and launch delays, everyone seems to have a data migration horror story to tell. While there is no magic pill or silver bullet to make data migration efforts flawless every time, there are a couple of ways to soothe the pain.
Data migration is an IT issue. Business users expect to have accurate, appropriately accessible data, but they don’t care about the details of how or where the data is stored. While there will be many business considerations to make throughout the data migration process, the planning and implementation of moving data from one system to another rests solely on IT’s shoulders. Business users must be engaged to identify the needs and use of data throughout the organization. The buy in of the business users will be critical because they will be the ones using the data once the migration is complete. However, just because the business users are engaged and consulted does not mean they are responsible for the success of the migration. The brass tacks of making the migration happen and ensuring the data is readily accessible is the responsibility of IT. The accountability for the success (or failure) of data migration is all in their hands. It is a huge responsibility not to be taken lightly.
Data Virtualization: Pain relief method #1, the red pill
Data virtualization allows IT to put a data services layer in between the data sources and business users. Rather than directly accessing data sources, consuming applications connect to the virtualization server and pull data from virtual views based on defined business terms. The data pulled from the sources can be cached at the virtualization server to provide persistent availability even if a data source has intermittent availability. If a data source changes or requires an upgrade for whatever reason, the change can happen in the background without affecting the business environment. Unexpected coding challenges are significantly reduced, if not eliminated, as the need to script rules and processes between every application and data source is no longer necessary.
When data needs to be migrated in a virtualized data environment, if there are unexpected hiccups, there is not a direct impact on the business environment. This removes many of the common causes for the IT data migration headache. It is important to note that data virtualization will not be answer to every data migration and integration problem. However, using data virtualization as part of a larger enterprise data management plan streamlines data access and integration and frees resources to work on other, more detailed (and often higher profile) initiatives.
Data Governance: Pain relief method #2, the blue pill
Data migration, by nature, is often project based at best. In fact, data migration is often considered a step of an overarching project. All too often efforts from multiple migration projects overlap and are duplicated resulting in unnecessary costs, missed deadlines and employee burnout. While there are definitely specific requirements for varying projects, much of the heavy lifting can be eased through well-defined, repeatable processes. Data migration projects steered by data governance programs are more successful, because data governance provides a framework for decisions, methods, procedures and strategic alignment of cross-functional efforts.
Data governance is a powerful pill as it not only knocks out the causes of the common data headaches, it helps prevent them. Data migration headaches are no exception. A finely tuned data governance program can reduce duplicate data throughout the organization, reduce errors in reporting and coding and reduce costs associated with poor data quality. The administrative framework and council of a well-defined data governance program support decisions, especially cross-functional decisions, with objectivity and consistency, allowing for decisions to be made quickly with greater accuracy and confidence. The data governance charter also supports and encourages business and IT collaboration as well as interdepartmental data sharing. Business rules, processes and data requirements are consolidated and architectural decisions are made from an enterprise view (rather than departmental) when managed under the direction of data governance. The structure, rigor and organization of data governance and management allow data migration projects throughout the organization to be completed using consistent, repeatable methods allowing the projects to be completed in less time, with greater accuracy and efficiency. Successful data governance rests in the hands of business and IT. So, while data governance is indeed a powerful pill, it is one everyone must take together to achieve desired results.
Data migration projects will continue to be full of surprises, but as with most headaches, if you can get relief for the worst symptoms, you gain the strength to take on the rest of the challenges. Take the red pill, the blue pill, or take them both whichever you choose, here’s to being able to turn on the lights and think clearly again.