A standard protocol for describing the evaluation of ecological models
Benjamin Planque, Johanna M. Aarflot, Lucie Buttay, JoLynn Carroll, Filippa Fransner, Cecilie Hansen, Bérengère Husson, Øystein Langangen, Ulf Lindstrøm, Torstein Pedersen, Raul Primicerio, Elliot Sivel, Morten D. Skogen, Evelyn Strombom, Leif Christian Stige, Øystein Varpe, Nigel G. Yoccoz
Abstract
Numerical models of ecological systems are increasingly used to address complex environmental and resource management questions. One challenge for scientists, managers, and stakeholders is to appraise how well suited these models are to answer questions of scientific or societal relevance, that is, to perform, communicate, or access transparent evaluations of ecological models. While there have been substantial developments to support standardised descriptions of ecological models, less has been done to standardise and to report model evaluation practices. We present here a general protocol designed to guide the reporting of model evaluation. The protocol is organised in three major parts: the objective(s) of the modelling application, the ecological patterns of relevance and the evaluation methodology proper, and is termed the OPE (objectives, patterns, evaluation) protocol. We present the 25 questions of the OPE protocol which address the many aspects of the evaluation process and then apply them to six case studies based on a diversity of ecological models. In addition to standardising and increasing the transparency of the model evaluation process, we find that going through the OPE protocol helps modellers to think more deeply about the evaluation of their models. From this last point, we suggest that it would be highly beneficial for modellers to consider the OPE early in the modelling process, in addition to using it as a reporting tool and as a reviewing tool.