Augmented robustness in home demand prediction: Integrating statistical loss function with enhanced cross-validation in machine learning hyperparameter optimisation

Abstract

• Propose robust ML tuning by combining mean and variance of loss function • Introduce Enhanced Multi-Fold Cross-Validation for stable data splitting • Evaluate model generalization on two additional household datasets • Achieve lower RMSE and tighter error bounds than baseline methods Sustainable forecasting of home energy demand (SFHED) is crucial for promoting energy efficiency, minimizing environmental impact, and optimizing resource allocation. Machine learning (ML) supports SFHED by identifying patterns and forecasting demand. However, conventional hyperparameter tuning methods often rely solely on minimizing average prediction errors, typically through fixed k-fold cross-validation, which overlooks error variability and limits model robustness. To address this limitation, we propose the Optimized Robust Hyperparameter Tuning for Machine Learning with Enhanced Multi-fold Cross-Validation (ORHT-ML-EMCV) framework. This method integrates statistical analysis of k-fold validation errors by incorporating their mean and variance into the optimization objective, enhancing robustness and generalizability. A weighting factor is introduced to balance accuracy and robustness, and its impact is evaluated across a range of values. A novel Enhanced Multi-Fold Cross-Validation (EMCV) technique is employed to automatically evaluate model performance across varying fold configurations without requiring a predefined k value, thereby reducing sensitivity to data splits. Using three evolutionary algorithms Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Differential Evolution (DE) we optimize two ensemble models: XGBoost and LightGBM. The optimization process minimizes both mean error and variance, with robustness assessed through cumulative distribution function (CDF) analyses. Experiments on three real-world residential datasets show the proposed method reduces worst-case Root Mean Square Error (RMSE) by up to 19.8% and narrows confidence intervals by up to 25%. Cross-household validations confirm strong generalization, achieving coefficient of determination (R²) of 0.946 and 0.972 on unseen homes. The framework offers a statistically grounded and efficient solution for robust energy forecasting. The optimum robust hyperparameter tuning of ML with enhanced multi-fold cross-validation.

Topics & Concepts