Foundation for unbiased cross-validation of spatio-temporal models for Species Distribution Modeling

Abstract

Evaluating the predictive performance of species distribution models (SDMs) under realistic deployment scenarios requires careful handling of spatial and temporal dependencies in the data. Cross-validation (CV) is the standard approach for model evaluation, but its design can strongly influence the validity of performance estimates. When SDMs are intended for spatial or temporal transfer, random CV can lead to overoptimistic performance estimates due to spatial autocorrelation (SAC) among neighboring observations. We benchmarked four machine learning algorithms (GBM, XGBoost, LightGBM, Random Forest) on two real-world presence–absence datasets — a temperate plant and an anadromous fish — under multiple CV designs: random, spatial, spatio-temporal, environmental, and forward-chaining. We evaluated two training-data usage strategies (LAST FOLD and RETRAIN) and applied extensive hyperparameter tuning within each CV scheme. Model skill was assessed on independent, out-of-time test sets using AUC, MAE, and correlation metrics. Random splits overstated AUC by up to 0.16 and yielded mean absolute errors (MAE) 70%–80% higher than spatially blocked alternatives. Blocking at the empirical SAC range largely mitigated this bias. Blocking at the empirical SAC range mitigated this bias across datasets. Training-data usage influenced evaluation outcomes: LAST FOLD tended to yield smaller validation–test discrepancies in SAC-prone settings, whereas RETRAIN provided higher test AUCs in cases with weaker SAC. Boosted ensembles generally performed best across spatially blocked CV designs for both datasets. We recommend a robust SDM workflow: (1) estimate SAC and construct spatial blocks accordingly; (2) tune hyperparameters using blocked cross-validation; (3) evaluate final models on external, out-of-time data. This pipeline enhances reliability for ecological forecasting under spatial and temporal shifts.

Topics & Concepts