When Hierarchical Federated Learning Meets Stochastic Game: Toward an Intelligent UAV Charging in Urban Prosumers
Luyao Zou, Md. Shirajum Munir, Yan Kyaw Tun, Sheikh Salman Hassan, Pyae Sone Aung, Choong Seon Hong
Abstract
Unmanned aerial vehicles (UAVs) nowadays are developing rapidly for various applications such as UAV taxis and delivery drones. However, the limited battery energy restricts the flight distance of the UAVs. Thus, urban prosumers equipped with drone recharge stations are introduced to provide charging services for the UAVs. In this article, first, a day-ahead energy scheduling problem for UAV charging-enabled urban prosumers is studied, where the objective is to maximize the overall energy satisfaction of the prosumers with ensuring the Quality of Service (QoS) of the charged UAVs. Specifically, to deal with the considered problem, we decompose it into two stages: 1) the day-ahead energy requirement data prediction stage and 2) energy scheduling stage per prosumer. Thus, second, a joint method based on hierarchical federated learning (HFL) on long short-term memory (LSTM) architecture (HFL-LSTM) and stochastic game-based multi-agent double deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning (MADDQN) with community agent-independent approach is proposed. In particular, the HFL-LSTM approach is leveraged to forecast each prosumer’s energy requirement data without centralized collecting local prosumers’ data such that to protect data privacy. Then, the stochastic game is adopted to analyze the formulated problem, aiming to find the Nash equilibrium (NE) strategy. Afterward, MADDQN with a community agent-independent method is utilized to achieve the best energy scheduling strategy per prosumer. Finally, the experimental results demonstrate the superiority of the proposed joint method that can achieve the lowest mean squared error with the value of 0.0152 and the highest energy satisfaction <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(36388)$ </tex-math></inline-formula> achieved by the NE policy compared with the benchmarks.