In today's information-driven digital economy, disaster recovery (DR) is often synonymous with data recovery. Companies depend on information to operate. If a critical application goes down, business screeches to a halt and revenue inevitably suffers. Companies that are complacent about DR planning take a huge risk: many businesses that lose their data due to disasters go out of business within five years.
And yet as recently as last fall - after Hurricane Katrina had devastated New Orleans and much of the Gulf Coast - many businesses still had their heads in the sand. An AT&T-commissioned survey found that one-third of 1200 companies surveyed had no disaster recovery plan. Furthermore, nearly a quarter of the companies surveyed had not updated their plans in the preceding 12 months.1
In a recent article about organizational resiliency, Harvard Business Review Senior Editor Diane L. Coutu opines: "[Resilient businesses] train [themselves] how to survive before the fact." 2 Coutu points to Morgan Stanley's "hard-nosed realism" in disaster planning as a perfect example. Morgan Stanley, the biggest tenant of the Trade Center Towers, had begun preparing for a major terrorist attack after the 1993 Trade Center bombing. Management invested in three backup centers as part of a larger corporate recovery plan. The day after the September 11 attacks, the company was serving customers.
As with many things, preparation is the key ingredient to DR. Firms like Morgan Stanley recover, even thrive, because they prepare. A month before Katrina pounded the Gulf Coast, Alfa Insurance invested in self-contained mobile recovery units equipped with power as well as voice and data communication systems. After the hurricane hit, adjusters were immediately able to conduct business with policyholders in Mississippi and other areas.3 Without the mobile units, Alfa would have lost sales, and their customers would have been left in the dark.
Adams and Reese, LLP was also ahead of the storm. The New Orleans law firm had already instituted scheduled system backups and had moved backup tapes to an off-site location long before Katrina hit. When Katrina did hit, their email system went down for only 15 minutes, thanks to the third-party emergency mail service the firm engaged following the events of September 11.4
No Template Plan
Disaster recovery plans are heterogeneous by definition. Two companies in the same industry - diverse in size, operations and systems - require two different recovery strategies. Industry classification also dictates which systems should be fixed first and how quickly they must come back online. An online florist relies on e-mail to verify orders and shipping, while an office supply company needs its customer database and phone system to stay in business.
In general, priorities in restoring a business follow an order similar to the following: 1) people, 2) power, 3) hardware, 4) software and 5) data. In other words, employees - at least those who are critical to core functions - should be brought back online before hardware, software and data. What good is equipment and data when no one is able to work with them? By the same reasoning, hardware and software require power to operate, while data requires not only employees to work with it, but hardware and software with which it can be managed, stored and accessed.
There is no one-size-fits-all DR plan - different disasters require different recovery plans. However, there are some basic steps every company should follow to ensure they are prepared. The first step is identifying the most likely natural disaster scenarios; this will dictate how preparations will evolve. California doesn't have tornados, but much of the state sits on active fault lines, making earthquakes a likely potential disaster. Twisters are commonplace in the mid-western United States, while seasonal hurricanes batter the Atlantic and Gulf Coasts. Note: Companies operating overseas should perform independent assessments of the mostly likely regional threats.
The next step is categorizing business functions and systems in order of importance. Which are critical to revenue? Which are important but can wait a week or two to be restored? If a function isn't mission-critical, is it required for normal operations, or can you conduct business without it? Hard questions must be asked now to allow for effective crisis management when there simply isn't time to consider all elements to a decision.
Next, define a minimally acceptable timeframe for recovery for each system and an acceptable amount of data loss for your organization. This is a critical step, as it will have a significant impact on budget, preparation timelines and pre-placement of people and equipment. The metrics for downtime and data loss will vary by customer type, size and industry. Some companies (e.g., banks and other financial firms) require zero downtime and loss of data; others (e.g., some manufacturing firms) are less impacted if systems go down for a while and can survive some data loss.
Bear in mind that the more business functions placed in the mission-critical category, the more costly the data recovery - so plan judiciously. If you're considering outsourcing data recovery, include vendors and fees in your DR budget; also factor in costs over several years - different DR approaches have different cost structures over one, two or three years. Your plan should also reflect any last-resort recovery vendor costs in the event of a disaster impacting both primary and backup systems.
While seemingly obvious, also make sure any contracts with DR vendors contain service level agreements (SLAs), which reflect the DR plan. If all mission-critical services need to be restored within a week of a disaster, all relevant contracts should contain an agreement by each vendor to restore their portion of the services to the extent and in whatever timeframe needed to meet such a requirement. All too, often businesses focus only on the terms of the biggest vendor contracts when smaller vendors may be contributing services on which these bigger vendors depend. Failing to take this factor into account could render vendor SLAs worthless.
