Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning

• Published in: Physics of fluids (1994) 2020-05, Vol.32 (5)
• Main Authors: Tang, Hongwei, Rabault, Jean, Kuhnle, Alexander, Wang, Yan, Wang, Tongguang
• Format: Article
• Language: English
• Subjects: Active control; Algorithms; Artificial neural networks; Computational fluid dynamics; Computer simulation; Drag reduction; Flow control; Fluid dynamics; Fluid flow; Interpolation; Machine learning; Mass flow rate; Neural networks; Optimization; Physics; Reynolds number; Robust control; Synthetic jets; Two dimensional flow
• ISSN: 1070-6631; 1089-7666
• DOI: 10.1063/5.0006492

This paper focuses on the active flow control of a computational fluid dynamics simulation over a range of Reynolds numbers using deep reinforcement learning (DRL). More precisely, the proximal policy optimization (PPO) method is used to control the mass flow rate of four synthetic jets symmetrically located on the upper and lower sides of a cylinder immersed in a two-dimensional flow domain. The learning environment supports four flow configurations with Reynolds numbers 100, 200, 300, and 400, respectively. A new smoothing interpolation function is proposed to help the PPO algorithm learn to set continuous actions, which is of great importance to effectively suppress problematic jumps in lift and allow a better convergence for the training process. It is shown that the DRL controller is able to significantly reduce the lift and drag fluctuations and actively reduce the drag by ∼5.7%, 21.6%, 32.7%, and 38.7%, at Re = 100, 200, 300, and 400, respectively. More importantly, it can also effectively reduce drag for any previously unseen value of the Reynolds number between 60 and 400. This highlights the generalization ability of deep neural networks and is an important milestone toward the development of practical applications of DRL to active flow control.