What saved the three Apollo 13 astronauts from utter disaster? Keep in mind that 4 days into their mission in April 1970, they suffered an explosion from an oxygen tank causing severe damage to the service module.
Besides for divine providence, what saved them comes down to three factors:
- All 3 astronauts were well trained and skilled in understanding the technology in their spacecraft and how to run and repair it
- NASA kept copious documentation on all aspects of the spacecraft and playbooks for dealing with common malfunctions
- On board the spacecraft were redundant systems and tools that the astronauts could use with guidance from Houston
Now if you compare those items to what the average security team has, you might find that items 1 and 3 are likely accounted for. In other words, the average security team has talented folks with skills who understand how to use their tools.
But let’s face it, the average security team doesn’t keep documentation. At least not as it concerns day to day operations, playbooks and use cases. And not having documentation almost certainly would have proven disastrous for Apollo 13. And it also forces security teams to work through different permutations of common tasks starting from scratch each time an issue arrises.
The absence of documented processes often times leads to common tasks blowing up into larger issues of their own.
It was commonly reported that the Target breach of 2013 could have been stopped much earlier in the attack lifecycle but the security team didn’t have a common process or playbook for handling the security event that was observed.
Instead, the event was ignored for weeks until the company started detecting the fraudulent transactions.
So how can we help our security teams do better?
Ask yourself and your team what their common scenarios are and if their processes are adequately documented. For example, do they have a process for documenting forensic incident details for tracking purposes? Does the document describe how to create a timeline, and what artifacts need to be recorded?
How well documented are your applications? Do you know which hosts are accessed by your customers? What security stack is deployed to each of those hosts?
What about scenarios they don’t see often, but when they happen you don’t want the team to have to stop and think about how to proceed? As an example, how would they deploy technology if you brought in a vendor to investigate a major incident? Is that documented?
This isn’t an exciting or shiny object.
But neither is any other form of practice and training that make professionals or astronauts better and safer at what they do.