Hope is a great quality for a baseball team in a pennant race to have, but it is not very good as a business continuity strategy. This is why it is critical that organizations are conducting regular disaster recovery tests of their information technology systems, otherwise known as DR tests, or IT/DR tests.
These tests can inform you whether you are actually capable of recovering your systems in the event of a disruption and what issues are likely to come up while you are trying to do so. They reveal capability and identify gaps.
In some industries, IT/DR tests might not simply be a good idea; they might be required. This is often the case, for example, with companies that must satisfy FDA requirements or meet SOX reporting mandates.
For all of these reasons, it’s well worth it for your team to learn how to conduct such tests properly and ensure they are well-designed and meet their goals. To help you do this, today we are sharing our “8 Dos and 1 Don’t for Conducting IT/DR Exercises.”
Here are the eight things we believe you and your team should do when conducting IT/DR exercises:
Why are you conducting the exercise? What do you hope to find out? There are many good reasons for conducting an IT/DR exercise, and one test can serve multiple purposes. Here are a few things you can accomplish by performing disaster recovery tests:
There are several different kinds of IT/DR exercise. Which kind will you be conducting? Here are the main options:
Exercises can vary widely in scope. Some look to verify that you can bring up a single application (such as your inventory app or financial app). Others look at multiple apps or at the integration between apps, whether internal or external. Some exercises might look at the full lifecycle of a business process, for example, the order and inventory lifecycle, taking something from order to delivery to receipt and invoice.
Based on the scope, identify which resources (servers, applications, etc.) and support personnel you need to conduct the exercise.
In many cases, a test environment must be set up specifically for the test. It might be necessary to set up a special network, for example. The test environment ensures the exercise does not impact the production environment. (Note: Preparing the test environment is different from preparing your systems so they perform better during the exercise. That sort of preparation is discussed below, in Do #7.)
Try to make the test environment mirror as closely as possible the way the environment would be set up in a real event.
How much notification you provide depends on your goals and priorities. The less notification you give, the more the exercise will resemble a real disruption—and the more accurate a picture it will provide of your current recovery capability.
The more notification you provide, the more people will have the opportunity to fix problems before the test happens. These fixes might result in long-term improvements in resiliency. Organizations with mature IT/DR programs should be capable of performing very well on unannounced exercises. There are three basic levels of notification:
Decide whether in performing the exercise the company will follow the regular crisis communication plan and procedures or whether you will use a (pre-planned) special organization. The special organization might use a defined bridge line, email list, schedule of status calls, and so on.
And there is one thing you should definitely not do if you wish to conduct disaster recovery tests that give real insight into and identify gaps in your recovery program. It is:
We often see organizations that get good at doing one type of exercise then perform that one exercise over and over again, while persuading themselves they must be in great shape because they perform DR tests all the time and always do well. This would be like someone who goes to the gym every day but limits themselves to lifting dumbbells with their right hand.
Sure, their right bicep might be strong, but what about their aerobic capacity, flexibility, or core strength? A resilient IT/DR environment has a well-rounded fitness. This is why it’s important to not perform only one type of DR exercise. Mix it up!
Conducting IT/DR exercises is critical for gaining insight into the recoverability of your environment and increasing its resiliency. You might also be required to perform such exercises for audit reasons. By following the 8 Dos and 1 Don’t set out in this post, you can increase your chances of achieving an effective IT/DR test program.
For more information on some of the topics mentioned above, check out these recent MHA Consulting and BCMMETRICS blog posts: