OceanGym: GPU-Accelerated Underwater RL Simulator

• OceanGym is a GPU-accelerated simulation framework for underwater RL research, integrating multi-sensor simulation, physically accurate hydrodynamics, and high-throughput parallel execution.
• It features a modular architecture with an environment manager, dual-layer physics engine, and GPU scheduler to support both low-level control and high-level perception, planning, and decision-making.
• The platform achieves rapid training with batch sizes up to 4096 and simulation speeds of 700,000 steps/sec, facilitating robust sim-to-real transfer and standardized benchmarking.

OceanGym is a comprehensive, GPU-accelerated simulation and benchmarking framework for underwater embodied agents, specifically targeting reinforcement learning (RL) for autonomous underwater vehicles (AUVs) and multimodal embodied AI. It integrates physically accurate hydrodynamics, multi-sensor simulation, and high-throughput parallel environment execution to support both low-level control and high-level perception, planning, and decision-making under the extreme conditions characteristic of oceanic environments (Chu et al., 2024, Xue et al., 30 Sep 2025, Chu et al., 12 Mar 2025, Liu et al., 2022).

1. System Architecture and Simulation Core

OceanGym is characterized by a modular, highly parallelized architecture designed for large-scale RL experimentation. The platform comprises three interacting runtime modules and a high-level RL interface:

• Environment Manager: Orchestrates $N$ independent AUV instances within a single process, synchronously advancing their states with lock-step step() and reset() calls. Each vehicle maintains separate state buffers (pose $\eta = [x, y, z, \phi, \theta, \psi]^\top$, velocity $\nu = [u, v, w, p, q, r]^\top$) resident in contiguous blocks of GPU global memory.
• Physics Engine Layer: Constructed in two tiers—
  • Rigid-Body Solver: Employs PhysX (via NVIDIA Omniverse Isaac Sim or Unreal Engine) for integration of rigid-body dynamics (gravity, buoyancy, collision).
  • Hydrodynamics Kernel: A custom CUDA module calculates added-mass, Coriolis, centripetal, and damping forces across all AUVs in parallel. These hydrodynamic terms serve as external wrenches communicated to PhysX at each simulation step.
• GPU Scheduler and Memory Manager: Handles batched kernel launches, asynchronous stream execution, and pooling/reuse of GPU memory pages to minimize allocation overhead.
• RL API: Exposes a Gym-style Python interface with PyTorch/TorchRL–native tensor I/O, enabling zero-copy communication between simulator and policy.

OceanGym’s simulation kernel is built for maximum throughput. With batch sizes up to 4096, observed throughput reaches 700,000 steps/sec (≈10,000× real-time) at 93% GPU utilization (NVIDIA RTX 3060 reference), completing training runs for canonical tasks in under four minutes (Chu et al., 2024).

2. Dynamic Modeling and Hydrodynamics

OceanGym employs the classic six-degree-of-freedom (6-DOF) Fossen formulation for underwater vehicle dynamics, combining rigid-body and hydrodynamic effects. The equations of motion in North-East-Down coordinates are: $$\boldsymbol{\tau} = \underbrace{ \mathbf{M}_{RB}\,\dot{\boldsymbol{\nu}} + \mathbf{C}_{RB}(\boldsymbol{\nu})\,\boldsymbol{\nu} + \mathbf{g}_{RB}(\boldsymbol{\eta}) }_{\text{Rigid-body}} + \underbrace{ \mathbf{M}_A\,\dot{\boldsymbol{\nu}} + \mathbf{C}_A(\boldsymbol{\nu})\,\boldsymbol{\nu} + \mathbf{D}(\boldsymbol{\nu})\,\boldsymbol{\nu} + \mathbf{g}_A(\boldsymbol{\eta}) }_{\text{Hydrodynamic}}$$ where $\mathbf{M}_{RB}$ and $\mathbf{M}_A$ are rigid-body and added-mass inertia matrices, $\mathbf{C}_{RB}$ and $\mathbf{C}_A$ the Coriolis terms, $\mathbf{D}$ the nonlinear damping, and $\mathbf{g}_{RB}$, $\eta = [x, y, z, \phi, \theta, \psi]^\top$0 the gravitational/buoyant and hydrostatic restoring forces.

Hydrodynamic damping is typically modeled as: $\eta = [x, y, z, \phi, \theta, \psi]^\top$1 with coefficients identified from empirical or first-principles data. The architecture supports plug-and-play extension of the physics kernel, allocation of arbitrary inertial/hydrodynamic parameter sets, and injection of custom external fields (e.g., flow, current) (Chu et al., 2024, Chu et al., 12 Mar 2025).

Inspired by FishGym, a plausible extension for broader OceanGym use would be explicit two-way fluid-structure coupling—e.g., GPU Lattice Boltzmann IB kernels—facilitating articulated and deformable agent simulation (Liu et al., 2022).

3. Task Suite and Benchmarking Protocols

OceanGym features a library of standardized RL tasks and decision benchmarks for AUVs and embodied agents. The four canonical control tasks include:

• Station-Keeping: Maintain pose at a target waypoint. Reward:

$\eta = [x, y, z, \phi, \theta, \psi]^\top$2

• Circle Tracking: Track a circular trajectory, with state augmented by future reference points.
• Helical Tracking: Extension of circle tracking with a vertical velocity component, producing a 3D helix.
• Lemniscate Tracking: Follow a figure-eight trajectory (Lemniscate of Bernoulli) (Chu et al., 2024).

All tasks are implemented with full domain randomization on physical parameters (mass, drag ±10%, thruster latency ±0.05 s) and curriculum learning (relaxed reward thresholds tightened over 200 epochs).

Decision-making benchmarks (as in the 2025 comprehensive incarnation (Xue et al., 30 Sep 2025)) include perception-planning tasks using simulated AUVs with multi-modal sensors:

Task Domain	Objective	Sensory Inputs
Locate the Robot	Approach and identify target robot	RGB, Sonar
Inspect Oil Pipe	Identify and report pipeline defects	RGB, Sonar
Search for Aircraft	Traverse to and report on wreckage	RGB, Sonar
Docking	Identify platform and approach center	RGB, Sonar, Pose

A plausible implication is that this integrated suite enables systematic head-to-head evaluation of RL algorithms, perception models, and multi-modal embodied planning approaches under ocean-specific constraints.

4. Sensor Modeling and Multi-Modal Perception

OceanGym implements physically-plausible sensor simulation as a core benchmark feature. AUV models mount six RGB cameras (forward, backward, left, right, up, down) and synthetic forward-looking sonar units producing 2D intensity maps in the same orientations. Optical image formation includes PBR-based shading and spectral attenuation according to water depth (illumination severely limited beyond 100 m), while sonar simulation supports beamwidth and range-based noise for realism (Xue et al., 30 Sep 2025).

Agents process observations via memory-augmented policy networks (e.g., Multi-modal LLMs, MLLMs), which are tasked with end-to-end perception, memory integration, and sequential decision-making. Experimental results demonstrate that leading MLLMs exhibit a substantial performance gap to human experts, particularly under deep water/low-light or noisy sonar conditions (e.g., shallow mean perception accuracy $\eta = [x, y, z, \phi, \theta, \psi]^\top$3 for MLLMs vs. $\eta = [x, y, z, \phi, \theta, \psi]^\top$4 for humans; deep environment: $\eta = [x, y, z, \phi, \theta, \psi]^\top$5 for models vs. $\eta = [x, y, z, \phi, \theta, \psi]^\top$6 human) (Xue et al., 30 Sep 2025). Decision scores in long-horizon navigation/planning are similarly reduced, highlighting essential research challenges for robust embodied intelligence.

5. GPU Acceleration and Parallelism

Through batched kernel execution, asynchronous CUDA streams, and zero-copy PyTorch tensor integration, OceanGym achieves simulation throughputs that exceed real-time by several orders of magnitude. The combination of N-parallel simulation (batch sizes to 4096+), physics/hydrodynamics entirely resident on GPU, and efficient reuse of memory buffers enables 700,000 steps/sec in benchmark configurations (Chu et al., 2024). Training for standard RL tasks (policy forward/backward + environment rollouts) routinely completes in under four minutes.

Performance benchmarking on typical consumer GPUs (e.g., NVIDIA RTX 3060) indicates resource utilization exceeding 85% for RL workloads, with raw physics stepping peaking at 1.2 million sub-steps/sec. These figures position OceanGym on par with or beyond contemporary frameworks such as MarineGym (Chu et al., 12 Mar 2025). This acceleration is critical for sample efficiency in RL and supports curriculum learning, robust domain randomization, and rapid policy validation (Chu et al., 2024, Chu et al., 12 Mar 2025).

6. Extensibility and Sim-to-Real Transfer

OceanGym facilitates the extension to new vehicle models and task domains by parameterizing inertial and hydrodynamic properties in YAML, auto-generating associated CUDA kernels, and supporting modular sensor arrays. Custom RL tasks are implemented through new Gym-compatible reward/state encoders. To bridge the sim-to-real gap, it is recommended to:

• Employ extensive domain randomization (physical parameters, currents, sensor noise)
• Fine-tune policies in a supervised fashion on real vehicle data with safety constraints
• Expand sensor simulation to include multibeam sonar, Doppler velocity loggers, and pressure sensors
• Adopt curriculum strategies, beginning with simplified scenarios and progressively increasing environment complexity (Chu et al., 2024, Chu et al., 12 Mar 2025).

Empirical results show that policies trained with domain randomization in simulation achieve substantially lower deployment error in out-of-distribution real-world conditions (e.g., station-keeping error: $\eta = [x, y, z, \phi, \theta, \psi]^\top$7 m with DR vs. $\eta = [x, y, z, \phi, \theta, \psi]^\top$8 m without DR, evaluated on environments beyond training ranges) (Chu et al., 12 Mar 2025).

7. Comparative Context and Research Significance

OceanGym builds upon a lineage of simulation benchmarks for underwater robotics, subsuming and generalizing design elements from platforms such as MarineGym and FishGym. Whereas FishGym emphasizes two-way fluid-structure coupling for fish-like robots (GPU LBM/IB, articulated skeleton+skinning, articulated motion), and MarineGym advances RL throughput via GPU hydrodynamics for rigid-body UUVs (Liu et al., 2022, Chu et al., 12 Mar 2025), OceanGym offers unified, standardized APIs, comprehensive environment and sensor modeling, and multi-modal policy evaluation for both low-level control and high-level embodied AI (Xue et al., 30 Sep 2025, Chu et al., 2024).

A plausible implication is that OceanGym’s extensible, high-fidelity, and ultra-fast simulation enables not just scalable RL research but also rigorous evaluation of perception, planning, and robust memory architectures. Its growing task library and support for hierarchical/multimodal agents advance sim-to-real methodology and set a reproducible research standard for autonomous oceanographic systems.

---

References:

• (Chu et al., 2024) MarineGym: Accelerated Training for Underwater Vehicles with High-Fidelity RL Simulation
• (Xue et al., 30 Sep 2025) OceanGym: A Benchmark Environment for Underwater Embodied Agents
• (Chu et al., 12 Mar 2025) MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Robotics
• (Liu et al., 2022) FishGym: A High-Performance Physics-based Simulation Framework for Underwater Robot Learning