RealPDEBench: Sim-to-Real Scientific ML Benchmark

Updated 3 July 2026

RealPDEBench is a comprehensive benchmark that pairs real-world measurements with simulations to evaluate physics-informed ML models.
It standardizes tasks such as real-world training, simulation-only training, and sim-to-real fine-tuning while employing nine complementary evaluation metrics.
The benchmark highlights key insights on sim-to-real generalization, improved convergence with simulation pretraining, and maintenance of physical invariants.

RealPDEBench is a comprehensive benchmark designed for evaluating scientific ML models on complex physical systems, with a specific emphasis on sim-to-real generalization. It provides paired real-world measurements and high-fidelity numerical simulations across representative fluid dynamic and reactive flow scenarios, enabling systematic analysis of model accuracy, robustness, and physical fidelity beyond simulation-only workflows (Hu et al., 5 Jan 2026).

1. Scope and Motivation

The central motivation for RealPDEBench is the persistent sim-to-real gap in scientific ML: while models trained and validated only on simulated data can display low error in-silico, their real-world performance is hindered by the presence of measurement noise, unmodeled physics, numerical artifacts, and restricted observability—all factors not accounted for in most synthetic datasets. RealPDEBench addresses this challenge by assembling more than 700 experimentally measured trajectories (each exceeding 2,000 time steps) covering distinct scenarios with matched simulated datasets (Hu et al., 5 Jan 2026). By enabling direct quantitative comparison between simulated and experimental outcomes, RealPDEBench provides a foundation for rigorous development and comparison of physics-informed ML surrogates, operator networks, foundation models, and hybrid methods for partial differential equations (PDEs).

2. Datasets and Physical Systems

RealPDEBench features five real-world measured datasets, each precisely paired with a computational fluid dynamics (CFD) or reactive flow simulation under identical parameter/initial condition settings:

Scenario	Measurement Method	Simulation/Physical Model
Cylinder Wake Flow	PIV (400 Hz)	2D Incompressible Navier–Stokes, Immersed-Boundary
Controlled Cylinder	PIV (harmonic)	NS with moving-boundary IBM
Fluid–Structure (FSI)	PIV (500 Hz)	Coupled NS + 1-DOF oscillator
Foil Cross-Section	PIV (3D)	3D CFD (Navier–Stokes, IBM)
Thermoacoustic Combustion	OH* chemilum.	3D unsteady LES with EDC, detailed chemistry

Key details include spatial/temporal normalization, removal of erroneous vectors in PIV (Histogram and median filtering), precise synchronization between measurements and simulation, and support for different physics regimes such as turbulence, controlled actuation, FSI, three-dimensional geometry, and turbulent combustion. Measurement and simulation data are structured for direct algorithmic ingestion, facilitating end-to-end benchmarking (Hu et al., 5 Jan 2026).

3. Task Definitions and Experimental Protocols

RealPDEBench standardizes three primary tasks, all constituting a mapping $F: \mathcal{A} \times \Gamma \to \mathcal{U}$ :

Real-World Training: Models are trained on real-world trajectories $\{(a_r, \gamma) \to u_r\}$ to minimize MSE between predicted and measured system evolution.
Simulation-Only Training: Models are trained solely on simulated trajectories $\{(a_s, \gamma) \to u_s\}$ , with modality masking and noise injection to approximate real measurement conditions.
Sim-to-Real Fine-Tuning: Models are first pretrained on all simulated samples, then fine-tuned on limited real-world data. The pretraining and adaptation losses are weighted via $\mathcal{L}_\text{pre} = \mathcal{L}_\text{data}(\text{sim}) + \lambda\mathcal{L}_\text{data}(\text{real})$ , where $\lambda$ transitions from 0 to 1.

Long-range forecasting is evaluated via autoregressive protocols (feeding predictions back as input for several rounds) (Hu et al., 5 Jan 2026).

4. Evaluation Metrics

RealPDEBench employs nine complementary metrics to rigorously quantify both data-centric and physically relevant accuracy:

Root Mean Squared Error (RMSE): Averaged over all times/locations.
Mean Absolute Error (MAE).
Relative L2 Error (Rel L2) and Coefficient of Determination (R²).
Update Ratio: Ratio of fine-tuning to real-only training steps needed to reach a target RMSE.
Fourier Space RMSE (fRMSE): RMSE in spectral (wavenumber) bands, distinguishing low, mid, and high-frequency errors.
Frequency Error (FE): Measures discrepancy in dominant oscillatory (e.g., vortex shedding) frequencies using temporal FFTs.
Kinetic Energy Error (KE).
Mean Velocity Profile Error (MVPE): Averaged error over spatial probe profiles, quantifying mean-flow discrepancies (Hu et al., 5 Jan 2026).

These metrics ensure that models are not merely evaluated for pointwise error but are also scrutinized for preservation of key engineering and physical invariants (e.g., energy, coherent structures, periodicity).

5. Models and Baselines

A diverse set of ten baselines spanning five methodological categories is benchmarked:

Category	Methods
Traditional Surrogate	DMD (Dynamic Mode Decomposition)
CNN-based	U-Net, CNO (Convolutional Neural Operator), DeepONet
Spectral Operator	FNO (Fourier Neural Operator), WDNO (Wavelet Diffusion NO), MWT
Transformer-based	GK-Transformer (Galerkin), Transolver
Foundation Model	DPOT (Denoising Pre-trained Operator Transformer), small (30M) and large (509M)

Each baseline is evaluated on all tasks, using the official PyTorch codebase and adhering to well-documented hyperparameter and solver configurations (Hu et al., 5 Jan 2026).

6. Key Results and Insights

Several systematic findings emerge from RealPDEBench:

Sim-to-Real Gap: ML models trained purely on simulated data suffer significant degradation on real measurements. For example, on Cylinder Flow, Rel L₂ error for simulation-trained models is 0.075, compared to 0.071 for real-only trained; errors can increase by up to 70% for some metrics.
Benefit of Sim-Pretraining: Pretraining on abundant simulation data and fine-tuning with limited real data consistently yields both lower prediction error (5–15% RMSE improvement) and reduced convergence times (average Update Ratio ≈ 0.6, indicating 40% faster convergence).
Physics-Fidelity vs. Pixelwise Accuracy: Pure convolutional models (U-Net, CNO) tend to minimize pixelwise RMSE but often underperform in preserving spectral or physical invariants relative to spectral/wavelet or foundation models (FNO, MWT, DPOT).
Autoregressive Long-Horizon Evaluation: Convolutional architectures accumulate errors faster in long-term rollouts, while spectral and large foundation models preserve stability and mean-flow structure longer.
Frequency-Band Specificity: Certain models, such as CNO, demonstrate improved accuracy in high-frequency bands (on Foil data), correlating with specialized alias-free filtering mechanisms (Hu et al., 5 Jan 2026).

7. Research Impact, Integration with Other Benchmarks, and Future Directions

RealPDEBench complements and extends prior scientific ML benchmarks such as PDEBench (Takamoto et al., 2022) and PINNacle (Hao et al., 2023), which focus on simulated datasets and classical or PINN-based baselines. By integrating matched experimental data and addressing the sim-to-real challenge directly, RealPDEBench facilitates the development of adaptation algorithms (e.g., PhysGuard (Zhou et al., 15 Jun 2026)) that selectively preserve physics-critical model directions during fine-tuning.

Prominent experimental advances, such as Fisher-guided gradient projection, have demonstrated up to 32% reduction in low-frequency error and substantial improvements in R² under domain shift, ranking first in the majority of architecture–scenario–metric combinations in RealPDEBench (Zhou et al., 15 Jun 2026).

Planned expansions include new domains (e.g., electromagnetics, structural dynamics, multiphase flows), task-specific physics metrics (such as conservation errors for reactive/compositional systems), and principled domain-adaptation pipelines. The benchmark suite and datasets are accessible at https://realpdebench.github.io/ (Hu et al., 5 Jan 2026).

A plausible implication is that RealPDEBench will increasingly shape the evaluation protocol for any scientific ML surrogate model intended for real-world deployment, providing a structured testbed for robust, generalizable, physics- and data-driven modeling.