Deep reinforcement learning-based energy management for design and control of off-grid renewable microgrids with dual-battery storage
Wajid Khan, Feng Renhai, Abdul Aziz, Muhammad Zain Yousaf, Zhi Cai, Muhammad Iqbal, Jiang Wang, Mustafa Abdullah, Mebratu Sintie Geremew
Abstract
Meeting the growing global electricity demand in remote and off-grid regions requires cost-effective and reliable power solutions that overcome the intermittency of renewable energy sources. This paper presents a comprehensive techno-economic optimization framework for the design and operation of off-grid hybrid renewable energy systems (HRES) integrating photovoltaic (PV), wind turbine, biomass generator, diesel backup, and a dual-chemistry hybrid battery energy storage system (HBESS) combining lithium-ion and nickel-iron batteries. A detailed mathematical modeling approach is employed to capture the nonlinear dynamics, stochastic renewable behavior, battery degradation, and temperature-adjusted component efficiencies. The system is formulated as a multi-objective mixed-integer nonlinear programming problem targeting the minimization of life cycle cost (LCC), levelized cost of energy (LCOE), and CO 2 emissions while satisfying reliability constraints such as loss of power supply probability (LPSP < 0.01). To solve the optimization problem, advanced metaheuristic algorithms—Particle Swarm Optimization (PSO), Genetic Algorithms (GA), Grey Wolf Optimizer (GWO), and Differential Evolution (DE), and Salp Swarm Algorithm (SSA)—and a Deep Q-Network (DQN)-based reinforcement learning energy management strategy are implemented and benchmarked. The proposed DQN-based controller demonstrates superior performance over conventional rule-based and static dispatch methods by maintaining more stable battery state-of-charge (SOC) profiles, reducing degradation, and enabling intelligent real-time decision-making. Simulation results based on realistic meteorological and demand profiles reveal that the integrated DQN and HBESS strategy reduces total LCC by over 20%, CO 2 emissions by up to 30%, and battery degradation costs by over 10% compared to baseline systems. The Salp Swarm Algorithm (SSA) achieves the fastest convergence and the highest-quality Pareto-optimal solutions among all metaheuristics evaluated. Sensitivity analysis identifies diesel price and interest rate as the most influential parameters on LCOE, while load shifting through aggressive demand-side management further minimizes battery usage, operating costs, and emissions. The proposed framework not only addresses key challenges in off-grid microgrid design but also provides a scalable and robust pathway for sustainable rural electrification using hybrid storage and intelligent control.