A novel attention-enhanced LLM approach for accurate power demand and generation forecasting

Abstract

Accurate forecasting of electricity demand and generation is crucial for efficient grid management and sustainable energy planning. While large language models (LLM) have shown promise in various fields, their application to time series forecasting presents challenges, including limited cross-channel information capture and the complexity of prompt design. In this study, we propose a novel framework that combines multiple attention mechanisms with LLM, enabling effective feature extraction from both target and non-target variables without the need for prompt engineering. We conducted extensive experiments using real-world electricity demand and generation data from multiple regions in Japan to evaluate the proposed model. The results demonstrate that our model outperforms state-of-the-art LLM-based and other time-series forecasting models in terms of electricity demand and generation forecast task, achieving better performance than the latest LLM-based models without using prompts or increasing model size. Compared with the Long short-term memory network (LSTM), the mean absolute error (MAE) is reduced by 20.8%. Compared with the previous time-series LLM, the proposed model reduces memory usage by 49.3% and shortens training time by 35.7%. Additionally, the proposed model exhibits superior generalization ability, maintaining high performance even in zero-shot learning scenarios. Compared with the LSTM, MAE on the four test datasets is reduced by 16.6%.

Topics & Concepts