Constrained Reinforcement Learning for Resource Allocation in Network Slicing
Yizhen Xu, Zhengyang Zhao, Peng Cheng, Zhuo Chen, Ming Ding, Branka Vucetic, Yonghui Li
Abstract
In network slicing, dynamic resource allocation is the key to network performance optimization. Deep reinforcement learning (DRL) is a promising method to exploit the dynamic features of network slicing by interacting with the environment. However, the existing DRL-based resource allocation solutions can only handle a discrete action space. In this letter, we tackle a general DRL-based resource allocation problem which considers a mixed action space including both discrete channel allocation and continuous energy harvesting time division, with the constraints of energy consumption and queue package length. We propose a novel DRL algorithm referred to as constrained discrete-continuous soft actor-critic (CDC-SAC) by redesigning the network architecture and policy learning process. Simulation results show that the proposed algorithm can achieve a significant performance improvement in terms of the total throughput with the strict constraints guarantee.