FD-Bench: Fair Benchmark for Fluid Simulation

• FD-Bench is a modular and reproducible benchmark that standardizes evaluation of data-driven fluid simulators using neural PDE solvers.
• It decomposes methods into spatial, temporal, and loss components to isolate performance differences and ensure fair comparisons with traditional CFD.
• It curates diverse, complex fluid simulation datasets and enforces consistent training and evaluation protocols to benchmark accuracy, efficiency, and generalization.

FD-Bench is a modular, comprehensive, and reproducible benchmark designed for evaluating data-driven fluid simulators, specifically neural PDE solvers, under a unified, fair, and systematic protocol. Addressing longstanding issues of comparability and reproducibility in the rapidly growing field of neural operator-based fluid dynamics, FD-Bench provides a standardized framework that disentangles spatial, temporal, and loss function modules, curates diverse fluid simulation datasets, and orchestrates large-scale, multi-dimensional benchmarking. This benchmark supports both the rigorous evaluation of new models and the extension of research via a user-friendly, open-source infrastructure (Wang et al., 25 May 2025).

1. Motivation and Benchmark Design Principles

Data-driven modeling of fluid dynamics has progressed rapidly, but comparison between neural PDE solvers is hindered by inconsistencies in dataset composition, discretization choices, boundary and initial conditions, and entangled architectural components. Unlike prior small-scale or single-focus datasets, FD-Bench introduces a comprehensive, modular evaluation protocol focused on fair, reproducible assessment:

• Modularization: Every method is decomposed into separate, swappable modules: spatial encoder (how spatial structure is encoded), temporal encoder (how temporal evolution/history is processed), and loss module (objective for training). This enables precise attribution of performance differences to architectural choices rather than confounding factors.
• Dataset Unification: All scenarios are discretized under a common framework, mitigating discrepancies in resolution, geometry, and BCs.
• Protocol Standardization: Training, testing, and evaluation occur under identical procedures—metrics, rollouts, conditions—across all baselines and all scenarios.
• Comparability to Traditional CFD: Neural solvers are systematically compared to high-fidelity numerical methods (e.g. pseudo-spectral solvers) under identical conditions.
• Extensibility: The modular codebase is open source, with standardized APIs and templates for new models, facilitating future development and broad community adoption (Wang et al., 25 May 2025).

2. Modular Architecture: Spatial, Temporal, and Loss Components

FD-Bench formalizes the structure of neural PDE solvers into three orthogonal axes, allowing for combinatorial exploration of design choices:

2.1 Spatial Encoders

The spatial encoder $\mathcal S_\theta$ maps field data sampled in the domain $\Omega\subset\mathbb{R}^d$ to feature representations. Supported paradigms:

• Fourier Neural Operator (FNO): Applies learnable spectral filters via fast Fourier transforms.
• Self-Attention: Implements transformer-style dot-product attention across spatial locations.
• Convolutional (CNN): Employs learned local stencils.
• Graph Convolutional: Utilizes adjacency matrices for mesh or graph-based domains.
• Reduced-Order Models (POD) and Implicit Neural Representations (e.g. MLPs/coordinate-based).

2.2 Temporal Encoders

Temporal encoding $\mathcal T_\theta$ supports a range of dynamical update strategies:

• Autoregression (AR): Predicts the next state from a history window.
• Next-step prediction: Single-frame input, rolled out sequentially.
• Temporal Bundling: Multi-input, multi-output blockwise prediction (multiple increments per step).
• Neural ODEs: Continuous-time evolution via parametric ODE solvers.
• Temporal Self-Attention: Attention modules act across time.

2.3 Loss Functions

Loss modules can be swapped to enforce different training objectives, including:

• Physical-variable MSE: Standard field-wise $\ell_2$ error.
• Diffusion-denoising loss: For generative models and uncertainty quantification.
• Flow-matching loss: Wasserstein-inspired vector field learning.
• Physics-informed residuals: Penalize discrete PDE residuals, analogously to PINNs.

This modularity enables systematic experimentation, e.g., evaluating the incremental value of self-attention vs. convolution for each temporal encoding method (Wang et al., 25 May 2025).

3. Curated Datasets and Coverage of Physical Phenomena

FD-Bench maintains ten scenarios spanning canonical and complex flows, including both grid-based and particle/mesh-based discretizations. All datasets are available in standardized NumPy/PyTorch formats with consistent train/validation/test splits and downsampled versions for multi-resolution studies:

Scenario	Type	Resolution/Structure
Incompressible Navier–Stokes	Vorticity, periodic	1200x1000x1x256x256
Compressible Navier–Stokes	ρ, u, v, p, periodic	42,000x4x4x128x128
Stochastic Incomp. N–S	u, v, random forcing	100x1000x2x128x128
Kolmogorov Flow	Forced, periodic	20,000x21x5x128x128
Diffusion-Reaction FitzHugh–Nagumo	Two-channel, Dirichlet	1,000x100x2x128x128
Taylor–Green Vortex (SPH)	Lagrangian, Re=100	204x126x2x10,000 (particles)
Reverse Poiseuille (SPH)	Lagrangian, compressible	1x30,000x2x12,800
Advection	Five velocity fields	1200x1000x2x256x256
Lid-Driven Cavity (SPH)	Particle, Re=100	1x20,000x11,236 (particles)
Burgers’ Equation	u, v channels	1200x1000x2x256x256

Initial conditions are sampled from analytic, Gaussian, or stochastic distributions as appropriate to each flow, supporting both in-distribution and zero-shot (held-out) regime testing (Wang et al., 25 May 2025, Baieri et al., 2023).

4. Evaluation Protocol and Performance Metrics

FD-Bench mandates identical training and testing procedures, with multi-faceted evaluation:

• Error Metrics: RMSE, normalized RMSE (nRMSE), and spectral RMSE (fRMSE) to capture structure-dependent accuracy. Spectral metrics diagnose model competency across different frequency bands.
• Stability Checks: Track divergence and boundary blow-up by monitoring $\|\hat u\|_\infty$ or other unphysical behaviors.
• Efficiency: Wall-time per rollout, peak memory usage, and GFLOPs.
• Physical Comparison: Each neural method (e.g., FNO) is directly benchmarked against traditional solvers at matched and lower spatial resolutions.
• Generalization: Models are evaluated under resolution shifts, initial condition/distribution shifts, and long-horizon extrapolation (error as a function of rollout length).

This protocol is designed to capture model trade-offs between accuracy, stability, efficiency, and robustness (Wang et al., 25 May 2025).

5. Benchmark Results and Comparative Findings

Systematic screening of 85 neural PDE models yields quantitative and qualitative insights:

• Spatial Encoding: Self-attention achieves the lowest RMSE and fRMSE on several flows (compressible N–S, diffusion–reaction, Kolmogorov) at moderate model size (~80M parameters), outperforming convolution and even FNO in most scenarios. FNO is runner-up, with similar accuracy but at lower compute cost.
• Temporal Encoding: Temporal bundling—multi-step, multi-increment prediction—outperforms standard autoregression, indicating that blockwise temporal structure improves both error and stability over long rollouts.
• Loss Functions: MSE remains the strongest baseline for predictive accuracy, while diffusion-denoising and flow-matching losses yield sharper or more physically consistent trajectories in some stochastic or uncertain regimes.
• Discretization: Eulerian grid-based FNOs outperform mesh-based (MeshGraphNets) and Lagrangian particle-based (GNS) methods in accumulative prediction errors, the latter showing weakest global structure retention.
• Neural vs. Classical Solvers: For several Reynolds numbers (50, 200, …, 2000) in incompressible N–S, FNO on $256^2$ grids attains up to $2\times$ lower long-horizon rollout error than a matched pseudo-spectral solver, and outperforms $128^2$, $64^2$ traditional solvers by factors of 2–5. FNO achieves a $10$–$\Omega\subset\mathbb{R}^d$0 speed-up in inference wall time at matched CFL constraints (Wang et al., 25 May 2025).

Resolution and initial condition generalization studies reveal that transformer-style (self-attention) models benefit most from parameter scaling, maintaining accuracy as depth and width increase, and being most robust to out-of-distribution regimes.

6. Reproducibility, Extensibility, and Practical Usage

FD-Bench prioritizes fair comparison and community contribution through the following features:

• Open Codebase: Python API for dataset/model/loss configuration; modular YAML-driven experiment orchestration.
• Readily Extendable: Simple templates for adding new modules; only two methods required per new spatial or temporal encoder.
• Integrated Data Handling: Scripts automate data loading, training, evaluation, and benchmarking against classical solvers.
• Community Guidance: Example notebooks and comprehensive test suites are provided for users to replicate and extend results (Wang et al., 25 May 2025).

Downloading and use are facilitated through Hugging Face and GitHub repositories, with standard data splits ensuring reproducibility and comparability across studies.

7. Current State of the Art and Research Outlook

FD-Bench’s leaderboard identifies the following combinations as state-of-the-art in predictive performance:

Spatial	Temporal	Loss	RMSE (Compressible N–S)	RMSE (Diff-Reaction)	RMSE (Kolmogorov)
Self-Attention	TemporalBundling	MSE	0.057	0.0067	0.0030
Fourier (FNO)	TemporalBundling	MSE	0.135	0.0093	0.0076

Models using generative (diffusion/flow-matching) losses yield higher prediction RMSE but exhibit better uncertainty and stochasticity modeling. Physics-informed (residual/PINN) loss methods enforce PDE constraints exactly, making them suitable for inverse and control applications, though with higher prediction error.

The modular design additionally exposes research direction: transformer architectures, more efficient continuous-time models, and hybrid deterministic–generative losses for fluid prediction. FD-Bench’s extensibility encourages development of new operators, incorporation of additional physics, and rigorous, head-to-head ablation studies.

• [FD-Bench: A Modular and Fair Benchmark for Data-driven Fluid Simulation, (Wang et al., 25 May 2025)]
• [Efficient Generation of Multimodal Fluid Simulation Data, (Baieri et al., 2023)]
• [CFDBench: A Large-Scale Benchmark for Machine Learning Methods in Fluid Dynamics, (Luo et al., 2023)]
• [FlowBench: A Large Scale Benchmark for Flow Simulation over Complex Geometries, (Tali et al., 2024)]