HydroGym: Modular RL for Flow Control

• HydroGym is a modular, solver-independent platform providing standardized RL environments for fluid dynamics challenges, spanning laminar flows to complex 3D turbulence.
• It integrates diverse fluid solvers—including LBM, FEM, and a differentiable JAX-based solver—to enable scalable, cross-domain reinforcement learning research.
• The platform extends to experimental testbeds and wearable biophysical sensing, promoting hardware-software co-design and reproducible benchmarking of flow control strategies.

HydroGym is a modular, solver-independent research platform offering standardized reinforcement learning (RL) environments for flow control in computational and experimental fluid dynamics. Designed to address the need for reproducible flow control benchmarks—spanning canonical laminar flows to 3D turbulence—HydroGym connects advanced RL algorithms with high-fidelity physics simulations, and provides pathways for laboratory integration and biophysical sensing, supporting scalable, cross-domain benchmarking and acceleration of RL-fluid mechanics research (Lagemann et al., 19 Dec 2025).

1. Platform Architecture and Components

HydroGym is organized around three principles: solver independence, comprehensive coverage of flow control problems, and scalable complexity across computational and hardware testbeds. Its environment stack exposes a Gymnasium-compatible API—supporting integration with standard RL pipelines and algorithms (e.g., PPO, DDPG, TD3)—and interfaces with three classes of fluid solvers:

• m-AIA Lattice Boltzmann (LBM): Non-differentiable, HPC-optimized, C++/MPI for scalable 2D/3D simulation.
• Firedrake FEM: Non-differentiable, Python-based, using PETSc and providing extensibility for custom PDEs and meshes.
• JAX-based solver: Fully differentiable, enabling gradient-enhanced RL (GPPO) for improved sample efficiency.

Environments are parameterized by geometry, Reynolds number range, actuation modalities (jets, surface rotation, thermal), and observation protocols (probe networks, field down-sampling). Multi-agent RL frameworks are supported for distributed control in large 3D domains. Solver backends implement a unified Python interface, allowing researchers to register custom environments and plug in RL algorithm implementations without modification to the environment code. All environments support minute-to-minute, episode-based interaction cycles suitable for RL routines (Lagemann et al., 19 Dec 2025).

2. Mathematical and Computational Foundations

HydroGym environments instantiate the governing equations of incompressible flow:

$$\frac{\partial \mathbf{u}}{\partial t} + (\mathbf{u} \cdot \nabla)\mathbf{u} = -\nabla p + \nu\nabla^2\mathbf{u} + \mathbf{f}, \qquad \nabla \cdot \mathbf{u} = 0$$

with alternate formulations for LBM and discrete weak-form (FEM) solvers. The reward functions are problem-specific but typically target drag/lift minimization (e.g., $r = -|C_D| - \omega(Re)|C_L|$ for bluff-body flows), stabilization of flow states, energy dissipation control, or direct agreement with target observables. Policy/value learning is performed via standard gradient-based updates, including the PPO clipped objective and deterministic policy gradients for DDPG/TD3. For environments implemented with JAX, HydroGym supports end-to-end differentiation, propagating RL gradients through the PDE integrator for accelerated convergence and sample efficiency gains with GPPO (Lagemann et al., 19 Dec 2025).

3. Benchmark Suite and Validation

HydroGym includes a set of 42 validated benchmark environments, grouping canonical flow control problems by geometry, dimensionality, and Reynolds regime:

Geometry	Dimension	Reynolds Range	Example Actuation	Observations
Cylinder	2D, 3D	100–3900	Jets, rotation	~30 pressure/velocity probes
Cavity	2D, 3D	4140–7500	Jets	~20 pressure probes
Pinball	2D, 3D	30–150	Surface rotation	~40 probes
Channel/Kolmogorov	2D, 3D	40–500 (2D), 180 (3D)	Body forcing	Down-sampled velocity field

Each environment is pre-validated against published DNS/LES/FEM results or empirical data, and supports a range of agent observation–action interfaces. This design enables rigorous benchmarking of flow control policies across physical regimes and established metrics. Multi-agent decomposition and full-field vs. probe-based sensing allow scalable investigations from single actuators to distributed controllers (Lagemann et al., 19 Dec 2025).

4. Hardware and Physical Testbeds

HydroGym principles extend to experimental facilities through the development of six-degree-of-freedom (6-DOF) hexapod testbeds for empirical hydrodynamic system identification:

• Motion Platform: Stewart hexapod with 6-UPU leg configuration, supporting translations (±0.34 m), rotations (up to ±48°), dynamic payloads up to 500 kg, and operational bandwidth 0.25–5 Hz (Ruiz et al., 28 Jan 2025).
• Sensors & Control: Six-axis load cells, high-fidelity servo actuation, motion capture (OptiTrack, ChArUco), and on-board hardware-in-the-loop digital twins. Data-driven modeling extracts added mass and damping matrices via multi-input multi-output (MIMO) multisine signal excitation and robust frequency response identification.
• Comparison to Planar Motion Mechanisms: The 6-DOF hexapod achieves orders-of-magnitude reduction in test time and enables full matrix identification (versus sequential, low-DOF, frequency-limited PMM approaches) (Ruiz et al., 28 Jan 2025).
• Scalability: Design pathways exist for large-scale HydroGym facilities (e.g., 3 m × 3 m × 100 m tow tanks), standardizing system identification protocols for cross-lab interoperability and high-throughput benchmarking.

5. Wearable Biophysical Sensing Integration

HydroGym is also designed for integration with real-time biophysical sensing, demonstrated by the HydroTrack prototype (Belabbaci et al., 2024):

• Hardware: 18-channel Triad spectroscopy sensor (410–940 nm), embedded in a smartwatch, with coordinated UV/visible/IR illumination and 28.6 nW/cm² precision.
• Signal Processing: Temporal amplification (Eulerian Video Magnification), 4th-order Butterworth filtration, per-channel aggregation, and feature extraction (raw, derivatives, rolling averages).
• Machine Learning: On-device 3-class hydration-state classification (XGBoost RF, up to 95% accuracy), with minute-scale updates and <1 s end-to-end latency.
• Data Integration: BLE/MQTT mesh protocols for multi-user, multi-device exercise monitoring; secure, time-synced, quantized absorbance outputs for TinyML at the edge.
• Scalability: Pathways for regression-based continuous hydration estimation, skin-tone adaptation, and motion-robust IMU filtering are documented for future integration in HydroGym-style platforms (Belabbaci et al., 2024).

6. Performance Evaluation, Transfer Learning, and Extensibility

Empirical studies in HydroGym report the following:

• Sample Efficiency Gains: Differentiable physics (JAX) enable up to 65% reduction in training episodes for controllers (e.g., 2D Kolmogorov, 3D channel flows) compared to classical RL (Lagemann et al., 19 Dec 2025).
• Emergent Control: RL agents autonomously discover physics-consistent strategies, including boundary-layer manipulation, wake reorganization (e.g., 93% drag reduction in pinball), acoustic feedback disruption, and vortex synchronization.
• Transferability: Controllers pre-trained at one Reynolds number, geometry, or dimensionality require ~50% fewer episodes for fine-tuning to novel regimes, demonstrating systematic policy transfer and data re-use efficiencies.
• Extensibility: HydroGym’s modular API allows straightforward registration of new flow environments, surrogate models, and RL algorithms, including Gymnax/PureJaxRL-based agents for differentiable control. Researchers can benchmark, extend, and compare RL strategies using a standardized, reproducible infrastructure (Lagemann et al., 19 Dec 2025).

7. Integration Pathways and Future Directions

HydroGym establishes the foundation for a scalable, cross-modal fluid control research ecosystem:

• Standardization: Provides unified interfaces, experiment management, and benchmark datasets for computational, experimental, and wearable flow sensing systems.
• Cross-Domain Research: Enables comparative evaluation and transfer learning between simulation-only, hardware-in-the-loop, and field-deployed fluid control platforms.
• Open-Source Availability: Platform code and environment definitions are openly available to the research community for rapid adoption and reproducibility in scientific studies (https://cl126162.github.io/TheHydroGymProject/).
• Scalability: Outlines architectural pathways for scaling to larger tow tanks, more complex hydrodynamic identification, and integration with emerging biophysical sensor modalities for holistic gym- and laboratory-scale multi-agent flow control (Lagemann et al., 19 Dec 2025, Ruiz et al., 28 Jan 2025, Belabbaci et al., 2024).

HydroGym thus constitutes an extensible, validated, and rigorously benchmarked platform for advanced reinforcement learning in fluid dynamics—facilitating reproducible research, hardware-software co-design, and the systematic development of state-of-the-art flow control methods.