Edge-Assisted Spectrum Sharing for Freshness-Aware Industrial Wireless Networks: A Learning-Based Approach
Mingyan Li, Cailian Chen, Huaqing Wu, Xinping Guan, Xuemin Shen
Abstract
Information freshness is essential to industrial wireless networks (IWNs) and can be quantified by the age-of-information (AoI) metric. This paper addresses an AoI-aware spectrum sharing (AgeS) problem in IWNs, where multiple device-to-device (D2D) links opportunistically access the spectrum to satisfy their AoI constraints while maximizing primal links’ throughput. Particularly, we orchestrate the access of D2D links in a distributed manner. Since distributed scheduling results in incomplete observation, D2D links share the spectrum with uncertainty on the transmission environment. Therefore, we propose a distributed scheduling scheme, called D-age, to deal with the transmission uncertainty in the AgeS problem, where an adaptation of actor-critic method is adopted with AoI constraints tackled in the dual domain. To address the non-stationary environment and multi-agent credit assignment issue, cooperative multi-agent reinforcement learning (MARL) approach is developed, where multiple local actors are designed to guide D2D links to make real-time decisions via distributed scheduling policies, which are evaluated by an edge-assisted global critic with action-aware advantage functions. Integrated with graph attention networks (GATs), the critic selectively learns contextual information by assigning different importances to neighboring links, which enables the evaluation of scheduling policies in a scalable and computation-efficient manner. Theoretical guarantee of the time-averaged AoI constraints is provided and the effectiveness of D-age in terms of both AoI violation ratio and the capacity of primal links is demonstrated by simulation.