Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming
Kailong Lin, Helin Yang, Mengting Zheng, Liang Xiao, Chongwen Huang, Dusit Niyato
Abstract
This paper proposes an aerial reconfigurable intelligent surface (RIS)-assisted maritime communication network (MCN) against jamming, where a hybrid RIS mounted on an unmanned aerial vehicle (UAV) is deployed to adjust and amplify signals as a communication relay. This paper aims to maximize the system energy efficiency (EE) considering jamming noise and quality of service (QoS) constraints of maritime users (MUs) by jointly optimizing resource scheduling of hybrid UAV-RIS including phase shift factor, amplitude coefficient, active RIS ratio, and transmit power of MUs. Due to dynamic and unknown environments, we design a novel approach based on penalized deep reinforcement learning (DRL) to solve the optimization problem, in which a penalized point policy difference authentic boundary proximal policy optimization (P3D-ABPPO) approach is proposed to enhance the learning capacity and system EE performance. Simulation results demonstrate that our proposed P3D-ABPPO-based hybrid UAV-RIS resource scheduling approach can significantly improve the system EE compared with other traditional DRL approaches. For example, the proposed P3D-ABPPO approach achieves system EE improvements of 10.38% compared with the PPO approach.