AoI-Aware Joint Spectrum and Power Allocation for Internet of Vehicles: A Trust Region Policy Optimization-Based Approach
Nuoheng Peng, Yan Lin, Yijin Zhang, Jun Li
Abstract
In Internet of Vehicles (IoV), information freshness is a significant indicator to indemnify road traffic safety, which is measured by Age of Information (AoI). In this article, we consider the coexistence scenario of vehicular user pairs and cellular users, where the base station (BS) acts as an agent to allocate channels and transmit power for vehicular user pairs. With the goal of minimizing the sum of the average AoI of all links and the average power consumption of all vehicular user pairs, we formulate this optimization problem as a discrete-time Markov decision process (MDP) problem and adopt the trust region policy optimization (TRPO) algorithm, which has the advantage of fast convergence and high stability. Then, an AoI-aware joint spectrum and power dynamic allocation scheme based on the TRPO algorithm is proposed. Simulation results show that the TRPO-based scheme significantly outperforms both the deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> network (DQN)-based scheme and the random scheme in terms of average cumulative reward, convergence speed, and stability.