Experimental deep reinforcement learning control of the aerofoil flow separation with a plasma actuator

In this study, experimental deep reinforcement learning (DRL) control of aerofoil flow separation is conducted at a chord-based Reynolds number of italic Re Subscript c Baseline equals 1.5 times 10 Superscript 5 Re c = 1.5 × 10 5 ${\textit{Re}}_c=1.5\times 10^5$ . A dielectric barrier discharge plasma actuator mounted at the leading edge and a hotwire placed in the separated shear layer act as the flow disturber and the state monitor, respectively. The closed-loop control law is parameterised by a radial basis function network and executed in real time on a field-programmable gate array at 1 kHz. With the aid of a deep deterministic policy gradient algorithm, a satisfying closed-loop control strategy can be derived in less than one minute, and the resulting lift coefficient increment (21 %) is similar to that achieved by the best open-loop control. For stable and effective DRL control, the sensor should be placed at the position with strongest velocity fluctuation, and both time delay and cashing period should be accounted in the reward design. Incorporating historical sensor measurements are beneficial to DRL control, and the optimal history length is approximately equal to the ratio of the local Taylor microscale to the control period. The final control law sought by DRL dictates that strong plasma actuation should only be applied when the flow separation region contracts to a minimum. Statistically, control benefits can be ascribed to both the increase of average reward at each cluster and the elevation of occurrence probabilities of high-reward clusters. Physically, suppression of the aerofoil flow separation is attributed to the excitation of large-scale shear layer vortices, which promotes the momentum exchange between the free stream and the reverse flow.