Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence

The development of a reliable subgrid-scale (SGS) model for large-eddy simulation (LES) is of great importance for many scientific and engineering applications. Recently, deep learning approaches have been tested for this purpose using high-fidelity data such as direct numerical simulation (DNS) in a supervised learning process. However, such data are generally not available in practice. Deep reinforcement learning (DRL) using only limited target statistics can be an alternative algorithm in which the training and testing of the model are conducted in the same LES environment. The DRL of turbulence modeling remains challenging owing to its chaotic nature, high dimensionality of the action space, and large computational cost. In the present study, we propose a physics-constrained DRL framework that can develop a deep neural network (DNN)-based SGS model for the LES of turbulent channel flow. The DRL models that produce the SGS stress were trained based on the local gradient of the filtered velocities. The developed SGS model automatically satisfies the reflectional invariance and wall boundary conditions without an extra training process so that DRL can quickly find the optimal policy. Furthermore, direct accumulation of reward, spatially and temporally correlated exploration, and the pre-training process are applied for the efficient and effective learning. In various environments, our DRL could discover SGS models that produce the viscous and Reynolds stress statistics perfectly consistent with the filtered DNS. By comparing various statistics obtained by the trained models and conventional SGS models, we present a possible interpretation of better performance of the DRL model.