Real-Time Melt Pool Homogenization Through Geometry-Informed Control in Laser Powder Bed Fusion Using Reinforcement Learning
Bumsoo Park, Alvin Chen, Sandipan Mishra
Abstract
This paper presents a real-time geometry-informed control strategy to homogenize melt pool measurements in laser powder bed fusion (L-PBF) using reinforcement learning. The learning control strategy incorporates geometric information of the scan path as well as in-situ melt pool measurements to compute the laser power signal for reducing in-process melt pool inhomogeneities. First, we design and validate a data-driven model to train the reinforcement learning agent in simulation, with the goal of reducing the amount of experimental data needed for training. Using this simulation-based training approach has the added benefit of avoiding unsafe or infeasible experiments, an issue that is often encountered in training the reinforcement learning agent. After training, the learned control strategy attenuates the 1-norm error by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf{37\%}$</tex-math> </inline-formula> and standard deviation by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf{39\%}$</tex-math> </inline-formula> in simulation. We then deploy this learned control strategy in an experimental test bed for a new scan geometry. In this test scenario, the policy achieves a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf{30\%}$</tex-math> </inline-formula> reduction in error, and a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf{36\%}$</tex-math> </inline-formula> reduction in melt pool signal variation, thereby illustrating the potential of reinforcement learning in real-time geometry-agnostic control for L-PBF. Finally, we demonstrate that the reinforcement learning agent delivers the same level of performance as a model-based feedforward controller with PID feedback, with 20 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> less computational time for a single geometry. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This work was motivated by the need to develop a practical control algorithm for L-PBF systems. Because L-PBF systems manufacture customized on-demand geometries, it is critical that the control strategy is extendable to and easily optimized for each geometry. Specifically, this effort develops an efficient and robust reinforcement learning control algorithm that can be used across novel part geometries, once trained. The control strategy is designed using a simulation-to-real approach, which is key for avoiding extensive training effort and avoids unsafe training experiments.