Surrogate model-based deep reinforcement learning for experimental study of active flow control of circular cylinder

The flow around a circular cylinder is a classical problem in fluid mechanics, and the reduction of drag and lift has been a long-standing research focus in flow control. In this study, we apply deep reinforcement learning (DRL) to intelligently determine suction flow rate on a circular cylinder model in wind tunnel, aiming to minimize aerodynamic forces while considering energy dissipation efficiency. However, DRL has been criticized for its low data utilization rate and long training period, leading to high experimental training cost. To address these issues, this study employs a surrogate model to optimize the reward function and hyperparameters, and this method is called SM-DRL. This SM-DRL method efficiently expedites the DRL training process, significantly reducing the experimental training cost. In addition, DRL training was conducted in a variable flow field, and the robustness of the obtained DRL model was tested. The results indicate that the DRL agent can determine the optimal control strategy, i.e., automatically select the optimal suction flow rate in terms of the incoming wind velocity, resulting in a significant reduction in lift fluctuations. For Reynolds number of 1.65×104, the reduction in lift fluctuations of the circular cylinder exceeds 50%.