Model Predictive Control Guided Reinforcement Learning Control Scheme
Huimin Xie, Xinghai Xu, Yuling Li, Wenjing Hong, Jia Shi
Abstract
Deep Reinforcement Learning (DRL) is an artificial intelligence technology that can complete decision-making tasks by interaction. It has been successfully applied to various games. However, there are still many challenges when this technique is applied to the industrial process control due to the low sample efficiency and the inability to deal with large time delay. In this paper, a novel Model Predictive Control (MPC) guided Reinforcement Learning Control (MP-RLC) scheme is proposed for the process control. In this scheme, Model predictive control is directly combined with Reinforcement Learning (RL) to guide the training process, thus greatly improving the sample efficiency of reinforcement learning and effectively solving the problem of time delay. The simulation results on both a third-order linear system and a nonlinear continuous stirred tank reactor (CSTR) system with large time delay demonstrate that this scheme can not only accelerate the training process but also improve the control performance, which is superior to both standalone RL and MPC schemes. The proposed approach may help to pave the way for DRL applied to industrial processes.