National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021
Johannes Bracher, Daniel Wolffram, Jannik Deuschel, Konstantin Görgen, Jakob Ketterer, Alexander Ullrich, Sam Abbott, Maria Vittoria Barbarossa, Dimitris Bertsimas, Sangeeta Bhatia, Marcin Bodych, Nikos I Bosse, Jan Pablo Burgard, Lauren Castro, Geoffrey Fairchild, Jochen Fiedler, Jan Fuhrmann, Sebastian Funk, Anna Gambin, Krzysztof Gogolewski, Stefan Heyder, Thomas Hotz, Yuri Kheifetz, Holger Kirsten, Tyll Krueger, Ekaterina Krymova, Neele Leithäuser, Michael Lingzhi Li, Jan H. Meinke, Błażej Miasojedow, Isaac Michaud, Jan Möhring, Pierre Nouvellet, Jedrzej Nowosielski, Tomasz Ożański, Maciej Radwan, Franciszek Rakowski, Markus Scholz, Soni Saksham, Ajitesh Srivastava, Tilmann Gneiting, Melanie Schienle
Abstract
BACKGROUND: During the COVID-19 pandemic there has been a strong interest in forecasts of the short-term development of epidemiological indicators to inform decision makers. In this study we evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland for the period from January through April 2021. METHODS: We evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland. These were issued by 15 different forecasting models, run by independent research teams. Moreover, we study the performance of combined ensemble forecasts. Evaluation of probabilistic forecasts is based on proper scoring rules, along with interval coverage proportions to assess calibration. The presented work is part of a pre-registered evaluation study. RESULTS: We find that many, though not all, models outperform a simple baseline model up to four weeks ahead for the considered targets. Ensemble methods show very good relative performance. The addressed time period is characterized by rather stable non-pharmaceutical interventions in both countries, making short-term predictions more straightforward than in previous periods. However, major trend changes in reported cases, like the rebound in cases due to the rise of the B.1.1.7 (Alpha) variant in March 2021, prove challenging to predict. CONCLUSIONS: Multi-model approaches can help to improve the performance of epidemiological forecasts. However, while death numbers can be predicted with some success based on current case and hospitalization data, predictability of case numbers remains low beyond quite short time horizons. Additional data sources including sequencing and mobility data, which were not extensively used in the present study, may help to improve performance.