The need for better statistical testing in data-driven energy technology modeling
Claudia Baumgartner, Rupert Way, Matthew C. Ives, J. Doyne Farmer
Abstract
Technology modeling is a vital part of developing and understanding energy system scenarios and policy, but it is challenging due to data limitations, deep uncertainty, and the complex social and technological dynamics involved in the evolution of energy systems. These difficulties are often compounded by unsound technology forecasting practice, including overfitting, data selection bias, and ad hoc assumptions, leading to unreliable conclusions. We flag several cases where this has been problematic and analyze in detail a recent model for predicting the pace of solar photovoltaic and wind energy deployment. We discuss general takeaways and provide suggestions for how statistical testing should be conducted to avoid such problems in the future and to quantify the reliability of forecasts.