Active flow control of a turbulent separation bubble through deep reinforcement learning

Abstract: The control efficacy of classical periodic forcing and deep reinforcement learning (DRL) is assessed for a turbulent separation bubble (TSB) at $Re_\tau=180$ on the upstream region before separation occurs. The TSB can resemble a separation phenomenon naturally arising in wings, and a successful reduction of the TSB can have practical implications in the reduction of the aviation carbon footprint. We find that the classical zero-net-mas-flux (ZNMF) periodic control is able to reduce the TSB by 15.7%. On the other hand, the DRL-based control achieves 25.3% reduction and provides a smoother control strategy while also being ZNMF. To the best of our knowledge, the current test case is the highest Reynolds-number flow that has been successfully controlled using DRL to this date. In future work, these results will be scaled to well-resolved large-eddy simulation grids. Furthermore, we provide details of our open-source CFD-DRL framework suited for the next generation of exascale computing machines.

• The paper investigates active flow control of turbulent separation bubbles using both classical periodic forcing and deep reinforcement learning within a LES computational fluid dynamics framework.
• Key results show that deep reinforcement learning achieves a significantly greater reduction in turbulent separation bubble size (25.3%) compared to classical periodic control (15.7%) in tested simulations.
• This study demonstrates the potential of DRL for developing adaptive, advanced flow control strategies that could improve aerodynamic efficiency and are scalable for future exascale computing applications.

Active Flow Control of a Turbulent Separation Bubble through Deep Reinforcement Learning

This paper presents a detailed investigation into the active flow control (AFC) of a turbulent separation bubble (TSB) utilizing deep reinforcement learning (DRL). Authored by a collaboration of researchers from institutions including the Barcelona Supercomputing Center and KTH Royal Institute of Technology, this study explores the comparative efficacy of classical periodic forcing and advanced DRL for TSB mitigation.

Summary of Objectives and Methodology

The primary objective is to mitigate the TSB, which can frequently occur on aircraft wings and impact fuel efficiency. The investigation uses two control strategies: conventional zero-net-mass-flux (ZNMF) periodic forcing, and a DRL strategy, both deployed at the upstream boundary layer. The experiment is executed within a computational fluid dynamics (CFD) framework using large-eddy simulations (LES) for a Reynolds number of $Re_{\tau} = 180$.

1. Classical Control: This method involves harmonic time-forcing of the wall-normal velocity. The results show a reduction in bubble size, providing a baseline for comparison with adaptive strategies.
2. DRL Control Framework: The paper applies a DRL paradigm where a neural network (NN) serves as an agent that learns effective control actions by interacting iteratively with the flow environment. The NN receives state information from the flow and outputs control actions aimed at optimizing a reward function based on bubble length reduction.

Results

Key findings reveal that DRL control surpasses the classical periodic control by a significant margin:

• Classical Periodic Control: Achieves a 15.7% reduction in the size of the TSB on the coarse grid, with a smaller figure on the fine grid due to the additional physical constraints manifest in finer calculations.
• DRL Control: Implements a more adaptive and optimal control scheme, recording a 25.3% reduction in TSB size on the coarse grid. This level of reduction is attributed to the model's ability to adjust its control actions dynamically, based on real-time observations of the flow state rather than predetermined oscillations.

The paper documents that the DRL strategy also ensures smoother control dynamics compared to the abrupt transitions in the classical control, potentially due to adjustments learned through multi-agent reinforcement scenarios and exploration of diverse flow states.

Implications and Future Directions

This study's results point towards significant potential for improving aerodynamic efficiencies in practical applications, such as reducing fuel consumption and emissions in aviation. The use of DRL introduces a transformative approach to flow control strategies, allowing for sophisticated, context-aware interventions that classical methods cannot achieve.

Looking ahead, the authors suggest scaling these promising results to larger simulations on more computationally demanding LES grids. Furthermore, the open-source CFD and DRL framework are crafted to be extensible and applicable with next-generation exascale computing systems, suggesting that practical applications could go beyond current laboratory-restricted scenarios.

This research highlights a critical step towards employing AI-driven techniques to solve complex fluid dynamics problems, fostering interdisciplinary synergy between AI research and applied physical sciences. The development of such integrated methodologies could pave the way for novel AFC strategies, further advancing the field of aerodynamics.