IoT Microservice Deployment in Edge-Cloud Hybrid Environment Using Reinforcement Learning
Lulu Chen, Yangchuan Xu, Zhihui Lu, Jie Wu, Keke Gai, Patrick C. K. Hung, Meikang Qiu
Abstract
The edge-cloud hybrid environment requires complex deployment strategies to enable the smart Internet-of-Things (IoT) system. However, current service deployment strategies use simple, generalized heuristics and ignore the heterogeneous characteristics in the edge-cloud hybrid environment. In this article, we devise a method to find a microservice-based service deployment strategy that can reduce the average waiting time of IoT devices in the hybrid environment. For this purpose, we first propose a microservice-based deployment problem (MSDP) based on the heterogeneous and dynamic characteristics in the edge-cloud hybrid environment, including heterogeneity of edge server capacities, dynamic geographical information of IoT devices, and changing device preference for applications and complex application structures. We then propose a multiple buffer deep deterministic policy gradient (MB_DDPG) to provide more preferable service deployment solutions. Our algorithm leverages reinforcement learning and neural network to learn a deployment strategy without any human instruction. Therefore, the service provider can make full use of limited resources to improve the Quality of Service (QoS). Finally, we implement MB_DDPG based on real-world data sets and some synthetic data, and we also implement another two algorithms, genetic algorithm and random algorithm, as a contrast. The experimental results demonstrate that MB_DDPG is able to learn a preferable strategy which, in terms of average waiting time, outperforms genetic algorithm and the random algorithm by 32% and 44%, respectively.