Incident Response: Preparation is the Name of the Game, Until Your Plans Change

By James Christiansen, Jeff Horne ·

The NHL and NBA playoffs are in full swing now with sports analysts and millions of crazed fans assessing and re-assessing every move the players make and critiquing every mistake. Things are no different when going through an incident response effort in the information security world. As we discussed in our recent webcast, “Incident Response: Giving the Advantage to the Hackers,” making the wrong decision based on incomplete or suspect information received in the middle of an incident response could be disastrous. And you’ll be judged by your executive team and board based on that one wrong move, just like that poor player who missed the winning shot at the buzzer or the one who couldn’t score on a penalty shot during overtime. Those serious mistakes are hard to forget and can ruin a reputation, whether you’re in sports or the business world. 

To help reduce the chance of making one of those possible career-ending decisions, you need to think like a world-class sports player. Here are some tips you can learn from the pros.

Before the Game
Coaches and players will study hours of film to understand their own, as well as opponents’ strengths and weaknesses. This helps coaches determine the best game plan that includes the most favorable player match-ups and the best plays to call. It also helps players understand what skills they need to improve upon. 

The proper preparation is critical to having even a chance of winning the game. This is no different in information security. You need to know, for example, what normal traffic on your network looks like. Don’t rush to dispose of those old system logs as you might need to reference them should an incident occur. And avoid tribal knowledge within IT. Be sure to diagram and document all systems and states—good and bad. 

Also, it’s absolutely critical to be sure you have the right tools, processes and people in place. Otherwise, an attacker could have your number at tip-off. Regularly exercising your incident response with real-world scenarios is critical to helping them build up their skills to address an incident when it occurs. An example might be simulating machines on your network being infected by Cryptolocker malware and running the team through a ransomware exercise. Another is to run legitimate hacks against your infrastructure prior to an incident to understand what happens. This will help your team make informed decisions and not assumptions that could turn into a costly mistake. And be sure to train your incident response team to handle the proper technologies that might be attacked, or be prepared to have the appropriate experts onsite to assist with the response efforts.

On Game Day
We’ve talked about the importance of preparation, but the biggest thing you should be prepared to handle is the unexpected. On game day, you might be dealing with situations you weren’t expecting. Maybe you have a key player hurt or the opponent is throwing out new plays you’ve never seen in any film. You feel like your back is against the wall, and you’re scrambling. So, what do you do? If what you planned isn’t working, you adjust your game plan. Call a time out so everyone can catch their breath and you mix up the plays.

A big mistake that happens during incident response efforts is the simple understanding that humans are involved, and they require one very important thing – sleep. Without enough sleep, people become part of the problem when responding to an incident. One way to address this is to cut a deal with a nearby hotel for temporary rooms for your team. Operate using the 16/6 rule mandating four hours of sleep at a time.

Another critical matter you’ll have to deal with on game day is reporting to your executive team. You must educate management to expect bad news, as incident response mitigations seldom have any good news during the first percent of the effort. This can be unsettling to your CEO, but helping him or her understand the process will make it easier on everyone.

After the Game
Just like those analysts and fans do after every game, coaches need to evaluate how things went, and so do you, after you deal with an attack. What went right and what went wrong? Do you have the right team in place to deal with another incident? Do you need to change processes or replace/add technologies? These are all questions you should be asking yourself, regardless of how well things went.

So, just remember that during a real-life attack, things will go wrong even when you have the best plan in place. But having a clear game plan, knowing how and when to change that plan, and re-evaluating that plan after the game can be the difference between a successful incident response effort and one that will have you looking for your next job.