Transfer Reinforcement Learning for Adaptive Task Offloading Over Distributed Edge Clouds
Kefan Shuai, Yiming Miao, Kai Hwang, Zhengdao Li
Abstract
In the big data era, resource-constrained mobile devices generate an overwhelmingly large amount of data with complex tasks that demand distributed execution. Offloading computation-intensive tasks to nearby edge clouds is promising to solve this problem. However, mobile end devices cannot handle heterogeneous or delay-sensitive tasks. These end devices are also energy constrained with weak adaptability to environment changes. To address and tackle these problems, we present a two-module <i>transfer reinforcement learning</i> (TRL) framework for adaptive task offloading. A domain adaptation module is used to align heterogeneous characteristics of mobile devices. The TRL makes offloading decisions with a <i>deep reinforcement learning</i> (DRL) module. We evaluate the performance of TRL through real-world experiments on edge clouds. Our experiment results show that TRL reduces the task processing time by a factor of 20% from using three well known DRL methods. Our method achieved (15.4 <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 40)% reduction in task drop rate over these methods. With domain adaptation, the TRL results in (50 <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 80)% reduction in model convergence time. These advantages in using the TRL framework make it appealing in real-life edge computing applications.