Challenges of Operational Weather Forecast Verification and Evaluation
Thomas C. Pagano, Barbara Casati, Stephanie Landman, Nicholas Loveday, Robert Taggart, Elizabeth E. Ebert, Mohammadreza Khanarmuei, Tara Jensen, Marion Mittermaier, Helen M. Roberts, Steve Willington, Nigel Roberts, Mike Sowko, Gordon S. Strassberg, Charles Kluepfel, Timothy A. Bullock, David D. Turner, Florian Pappenberger, Neal Osborne, N Chris
Abstract
Abstract Operational agencies face significant challenges related to the verification and evaluation of weather forecasts. These challenges were investigated in a series of online workshops and polls engaging operational personnel from six countries. Five key themes emerged: inadequate verification approaches for both existing and emerging products; incomplete and uncertain observations; difficulties in accurately capturing users’ real-world experiences using simplified metrics; poor communication and understanding of forecasts and complex verification information; and institutional factors such as limited resources, evolving meteorologist roles, and concerns over reputational damage. We identify nearly 50 operationally relevant scientific questions and suggest calls to action. Addressing these needs includes designing forecast systems with verification as a central consideration, enhancing the availability of observations, and developing and adopting community software systems. Additionally, we propose the establishment of an international community comprising environmental and social science researchers, statisticians, verification practitioners, and users to provide sustained support for this collective endeavor.