Deep-Reinforcement-Learning-Based Joint Caching and Resources Allocation for Cooperative MEC
Wenqian Zhang, Guanglin Zhang, Shiwen Mao
Abstract
The emergence of new applications has led to a high demand for mobile-edge computing (MEC), which is a promising paradigm with a cloud-like architecture deployed at the network edge to provide computation and storage services to mobile users (MUs). Since MEC servers have limited resources compared to the remote cloud, it is crucial to optimize resource allocation in MEC systems and balance the load among cooperating MEC servers. Caching application data for different types of computing services (CSs) at MEC servers can also be highly beneficial. In this article, we investigate the problem of hierarchical joint caching and resource allocation in a cooperative MEC system, which is formulated as an infinite-horizon cost minimization Markov decision process (MDP). To deal with the large state and action spaces, we decompose the problem into two coupled subproblems and develop a hierarchical reinforcement learning (HRL)-based solution. The lower layer uses the deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> network (DQN) to obtain service caching and workload offloading decisions, while the upper layer leverages DQN to obtain load balancing decisions among cooperative MEC servers. The feasibility and effectiveness of our proposed schemes are validated by our evaluation results.