Reliable adaptive edge-cloud collaborative DNN inference acceleration scheme combining computing and communication resources in optical networks
Shan Yin, Yurong Jiao, Chenyu You, Mengru Cai, Tianyu Jin, Shanguo Huang
Abstract
With the continuous development of the Artificial Intelligence of Things, deep neural network (DNN) models require a larger amount of computing capacity. The emerging edge-cloud collaboration architecture in optical networks is proposed as an effective solution, which combines edge computing with cloud computing to provide faster response and reduce the cloud load for compute-intensive tasks. The multi-layered DNN model can be divided into subtasks that are offloaded to edge and cloud servers for computation in this architecture. In addition, as bearer networks for computing capacity, once a server or link in optical networks fails, a large amount of data can be lost, so the robust reliability of the edge-cloud collaborative optical networks is very important. To solve the above problems, we design a reliable adaptive edge-cloud collaborative DNN inference acceleration scheme (RACAI) combining computing and communication resources. We formulate the RACAI into a mixed integer linear programming model and develop a multi-agent deep reinforcement learning algorithm (MADRL-RACIA) to jointly optimize DNN task partitioning, offloading, and protection. The simulation results show that compared with the benchmark schemes, the proposed MADRL-RACIA can provide a guarantee of reliability for more tasks under latency constraints and reduce the blocking probability.