Model Predictive and Reinforcement Learning Methods for Active Flow Control of an Airfoil with Dual-point Excitation of Plasma Actuators

Abstract: This paper presents an analysis of Model Predictive Control (MPC) and Reinforcement Learning (RL) approaches for active flow control over a NACA (National Advisory Committee for Aeronautics) 4412 airfoil around the static stall condition at a Reynolds number of 4*10^5. The Reynolds Averaged Navier-Stokes (RANS) equations with the Scale-Adaptive Simulation (SAS) turbulence model are utilized for the numerical simulations. The dielectric barrier discharge (DBD) plasma actuators were employed in dual excitation mode for flow separation control. The study systematically evaluates adaptive MPC, temporal difference reinforcement learning (TDRL), and deep Q-learning (DQL) based on optimizing the excitation frequency and expediting the time to identify stable conditions. Moreover, an integrated approach that combines signal processing with DQL is examined. The results demonstrate that while MPC and RL significantly improve flow control, RL approaches offer superior adaptability and performance. In optimal conditions, a lift coefficient of around 1.619 was achieved within less than 2.5 seconds with an excitation frequency of 100 or 200 Hz. This research highlights that RL-based approaches could perform better in flow control applications than MPC.