How Ready is Your Ready? Assessing the Usability of Incident Response Playbook Frameworks
Rock Stevens, Daniel Votipka, Josiah Dykstra, Fernando Tomlinson, Erin Quartararo, Colin R Ahern, Michelle L. Mazurek
Abstract
Incident response playbooks provide step-by-step guidelines to help security operations personnel quickly respond to specific threat scenarios. Although playbooks are common in the security industry, they have not been empirically evaluated for effectiveness. This paper takes a first step toward measuring playbooks and the frameworks used to design them, using two studies conducted in an enterprise environment. In the first study, twelve security professionals created two playbooks each, using two standard playbook design frameworks; the resulting playbooks were evaluated by experts for accuracy. In the second, we observed five personnel using the created playbooks in no-notice threat exercises within a live security-operations center. We find that playbooks can help simplify and support incident response efforts. However, playbooks designed using the frameworks we examined often lack sufficient detail for real-world use, particularly for more junior technicians. We provide recommendations for improving playbooks, playbook frameworks, and organizational processes surrounding playbook use.