Taxonomy of purposes, methods, and recommendations for vulnerability analysis

Abstract

Vulnerability analysis is an emerging technique that discovers concise descriptions of the conditions that lead to decision-relevant outcomes (i.e., scenarios) by applying machine learning methods to a large ensemble of simulation model runs. This review organizes vulnerability analysis methods into a taxonomy and compares them in terms of interpretability, flexibility, and accuracy. Our review contextualizes interpretability in terms of five purposes for vulnerability analysis, such as adaptation systems and choosing between policies. We make recommendations for designing a vulnerability analysis that is interpretable for a specific purpose. Furthermore, a numerical experiment demonstrates how methods can be compared based on interpretability and accuracy. Several research opportunities are identified, including new developments in machine learning that could reduce computing requirements and improve interpretability. Throughout the review, a consistent example of reservoir operation policies in the Colorado River Basin illustrates the methods. • Vulnerability analysis discovers concise descriptions of conditions that lead to decision-relevant performance outcomes. • Performance outcomes are binary, multi-class, or continuous, and methods are sorted by interpretability and flexibility. • Recommendation: use methods that maximize interpretability subject to accuracy requirements for the decision context.

Topics & Concepts