ClimateFiller: A Python framework for climate time series gap-filling and diagnosis based on artificial intelligence and multi-source reanalysis data
Chouaib El Hachimi, Salwa Belaqziz, Saïd Khabba, Youness Ousanouan, Badr-eddine Sebbar, Mohamed Hakim Kharrou, Abdelghani Chehbouni
Abstract
Clean weather time series is the primary ingredient for the successful modeling of any process in the soil-plant-atmosphere continuum. However, measured meteorological data are often associated with gaps due to various reasons, such as eventual sensor malfunctioning, power outages, and data transmission errors. Thus, meteorological data needs to be quality-controlled prior to any further processing. To this end, we developed a Python framework that automatically detects missing values and uses ERA5-Land reanalysis data and machine learning to fill in the gaps. The resulting RMSE values indicate a good consistency between the estimated and real in-situ data (0.73 °C, 26.07 W m2, and 11.57%, for air temperature, global solar radiation, and air relative humidity, respectively.). Additionally, the framework implements different data-driven methods to detect and fill in outliers and apply physics constraints to the data.