A Deep Reinforcement Learning Approach to Improve the Learning Performance in Process Control
Yaoyao Bao, Yuanming Zhu, Feng Qian
Abstract
Advanced model-based control methods have been widely used in industrial process control, but excellent performance requires regular maintenance of its model. Reinforcement learning can online update its policy through the observed data by interacting with the environment. Since a fast and stable learning process is required to improve the adaptability of the controller, we propose an improved deep deterministic actor critic predictor in this paper, where the immediate reward is separated from the action-value function to provide the actor with reliable gradient information at early stages. Then, an expectation form of policy gradient is developed based on the assumption that the state obeys the normal distribution. Simulation results show that the proposed algorithm achieves a more stable and faster learning procedure than those state-of-art deep reinforcement learning (DRL) algorithms. Meanwhile, the obtained policy achieves a more advantageous performance than the fine-tuned proportional integral and derivative (PID) and linear model predictive controllers, especially for those processes with nonlinearity. These indicate that the improved DRL controller has the potential to become an important tool in practical applications.