Closed-loop supersonic flow control with a high-speed experimental deep reinforcement learning framework
Haohua Zong, Yun Wu, Jinping Li, Zhi Su, Hua Liang
Abstract
Although active flow control based on deep reinforcement learning (DRL) has been demonstrated extensively in numerical environments, practical implementation of real-time DRL control in experiments remains challenging, largely because of the critical time requirement imposed on data acquisition and neural-network computation. In this study, a high-speed field-programmable gate array (FPGA) -based experimental DRL (FeDRL) control framework is developed, capable of achieving a control frequency of 1–10 kHz, two orders higher than that of the existing CPU-based framework (10 Hz). The feasibility of the FeDRL framework is tested in a rather challenging case of supersonic backward-facing step flow at Mach 2, with an array of plasma synthetic jets and a hot-wire acting as the actuator and sensor, respectively. The closed-loop control law is represented by a radial basis function network and optimised by a classical value-based algorithm (i.e. deep Q-network). Results show that, with only ten seconds of training, the agent is able to find a satisfying control law that increases the mixing in the shear layer by 21.2 %. Such a high training efficiency has never been reported in previous experiments (typical time cost: hours).