Decentralized Cooperative Caching and Offloading for Virtual Reality Task Based on GAN-Powered Multi-Agent Reinforcement Learning
Yang Yang, Lei Feng, Yao Sun, Yangyang Li, Fanqin Zhou, Wenjing Li, Shangguang Wang
Abstract
As a critical and prevalent service in future mobile networks, virtual reality (VR) is latency-sensitive and power-hungry, bringing out the optimization problem of trade-off among power saving, delay, and resource utilization. Content caching and render offloading are deemed as promising solutions to meet the stringent requirements of VR on data transmission speed and end-to-end latency. In this article, we propose a novel distributed computing framework based on multi-agent deep deterministic policy gradient (MADDPG) for joint optimizing terminal-cooperative caching and offloading for VR tasks. Since the individual VR user can hardly reach the optimal actions based on its limited local observed states and samples, MADDPG with centralized training and distributed execution is exploited to solve the above challenge. In addition, the generative adversarial network (GAN) is introduced to obtain experience-enhanced agents in the offline training phase and to achieve an optimal allocation to minimize energy consumption in the online inferring phase. The Nash equilibrium is proven in the case that the distribution of finite real VR data samples is well imitated and complemented by GAN. Numerical results demonstrate that our algorithm has significant superiorities in terms of convergence performance and energy consumption over other benchmarks.