Reinforcement Learning-Based Fed-Batch Optimization with Reaction Surrogate Model
Yan Ma, Zhenyu Wang, Iván Castillo, Ricardo Rendall, Rahul Bindlish, Brian Ashcraft, David Bentley, Michael G. Benton, José A. Romagnoli, Leo H. Chiang
Abstract
In this paper, we implement a framework which combines Reinforcement Learning (RL) based reaction optimization with first principle model and plant historical data of the reaction system. Here we employ a Long-Short-Term-Memory (LSTM) network for reaction surrogate modeling, and Proximal Policy Optimization (PPO) algorithm for the fed-batch optimization. The proposed reaction surrogate model combines simulation data with real plant data for an accurate and computationally efficient reaction simulation. Based on the surrogate model, the RL optimization result suggests maintaining an increased temperature setpoint and high reactant feed flow to maximize the product profits. The simulation results by following the RL profile suggests an estimate of 6.4% improvement of the product profits.