Training reinforcement learning-based controller using performance simulation of the laser remelting process

Surface topography modification is a common post-processing step used to enhance surface quality or to add micro/nano surface structures for various functionalities. Laser remelting (LRM) is a new technology aimed to resolve some of the shortcomings of traditional post-processing technologies. In LRM, a high-energy laser beam melts the top surface and offers control of the molten material flow through the control of laser power, scanning speed, melt pool size, and scan trajectory. By controlling the molten material flow, surface polishing by laser remelting (SP-LRM) can be used to produce finishes characterized by areal arithmetical mean height - Sa < 0.1 µm as well as surface microstructures. LRM is faster than traditional methods, creates no harmful byproducts, and allows for polishing high aspect ratio freeform surfaces without harming surrounding areas. The primary limiting factor for wide-rage industrial use of the LRM is represented by thermodynamic process instabilities that are associated with rapid melting and solidification. For instance, deep trenches or grinding marks in the initial surface can cause small deviations of local surface absorptivity to lead to laser melt pool instabilities. The high-speed nature of the LRM process combined with the complex thermodynamic phenomena makes process instabilities difficult to model, detect, predict, and control. This study aimed to use reinforcement learning (RL) in order to build a 'self-learning' controller capable of adapting to any material and control instabilities on-line and in real-time. Along these lines, the study will detail a preliminary application of two RL implementations involving SP-LRM performance simulation to demonstrate the implementation of a double dueling deep Q-network (DDDQN) controller for melt pool temperature distribution control. The results obtained showed that the 3-action implementation achieved 48% less variance and 35% increase in the mean value of the total reward collected compared with the 31-action implementation.