Striding Across Reynolds Numbers: Representation Geometry in Neural PDE Generalisation

Abstract: Cross-Reynolds generalisation in neural PDE solvers remains poorly characterised. On the canonical forced 2D Navier-Stokes benchmark, a trained Fourier Neural Operator reaches 46.68% relative L2 error under a 10x Reynolds-number shift, yet zero-forward-model retrieval baselines already improve to 41-42%. This suggests representation geometry as a major organising variable among the tested methods. We test this hypothesis through ConvAE-Relay, which matches states in a source-trained convolutional autoencoder latent space and borrows dynamics from a source-regime database, achieving 38.34+/-0.07% using only a source-regime database and no target-regime fitting, labels, or database entries. A 2x2 ablation isolates matching quality as dominant over the update rule. Oracle experiments confirm that source-regime dynamics directions remain transferable (cosine similarity ~0.84) when matching stays on-manifold; autoregressive drift is the primary bottleneck (~12 percentage points). From the learned-prediction side, a U-Net with multi-scale skip connections achieves 34.72+/-0.60%, consistent with the retrieval-side finding that local, multi-scale representations organise cross-Reynolds transfer among tested methods. All claims are scoped to this benchmark.

• The paper identifies representation geometry as the key factor affecting cross-Re generalisation in neural PDE solvers under a 10× Reynolds shift.
• It compares learned predictors like U-Net and FNO with retrieval-based methods, showing that optimized latent representations can significantly reduce prediction errors.
• The study highlights autoregressive drift and database coverage as primary challenges, setting clear engineering targets for improved PDE solver performance.

Striding Across Reynolds Numbers: Representation Geometry in Neural PDE Generalisation

Problem Motivation and Characterisation

Cross-parameter generalisation in neural PDE solvers is an outstanding challenge, particularly when solutions are extrapolated along physically meaningful axes such as the Reynolds number (Re), which governs the ratio of inertial to viscous forces in fluid dynamics. This work provides a systematic analysis of neural and retrieval-based PDE solvers under a $10{\times}$ Re shift on the forced 2D Navier–Stokes system, the canonical benchmark for neural operator research. The principal hypothesis is that the geometry of state representation is the organizing variable for cross-Re generalisation. The empirical investigations contrast learned predictors and database-driven (retrieval/analogue) methods, dissecting not only their aggregate accuracy but also the mechanistic reasons behind the observed transfer limitations.

Methodological Framework

Six approaches are evaluated: vanilla Fourier Neural Operator (FNO), a U-Net architecture with multi-scale skip connections, and four retrieval-based baselines varying in representation and update rule, including a newly introduced ConvAE-Relay method.

• Database Relay (ConvAE-Relay): This method indexes a source-regime trajectory database in a latent space constructed by training a convolutional autoencoder on source frames only. Predictions at OOD (target) Re are made by retrieving nearest states in latent space and borrowing their empirical transitions, with no learned forward dynamics or access to target data.
• U-Net Predictor: A conventional U-Net is trained as an autoregressive forward model, using context frames as input and predicting the next frame. The architecture incorporates local convolutional operations and skip pathways to encourage multi-scale feature preservation.
• Baselines: PCA relay (linear representation), $k$NN-copy in raw vorticity space, and pure FNO. All learned models are trained only on source ($\mathrm{Re}{\approx}1{,}000$) data, with no access to target labels or database entries.

  Figure 1: Summary of vorticity prediction visualizations, representation ablation results, and oracle bottleneck decomposition across methods at $\mathrm{Re}{=}10{,}000$.

Key Empirical Findings and Quantitative Results

The dominant finding is that representation geometry governs cross-Re generalisation quality, not the presence or absence of learned dynamics. The principal quantitative results are as follows:

• FNO plateaus at $46.68\%$ mean relative $L_2$ error under a $10{\times}$ Re shift ($\mathrm{Re}{=}10{,}000$), despite strong in-distribution performance.
• Zero-parameter analogue retrieval methods (PCA relay at $41.77\%$, $k$NN-copy at $k$0) outperform FNO, demonstrating that naive retrieval with no target training surpasses state-of-the-art spectral neural operators.
• ConvAE-Relay with source-trained convolutional latent space achieves $k$1 by changing only the matching representation—no target fitting, dynamics learning, or OOD database required.
• U-Net achieves $k$2, the lowest observed error, and its advantage is robust across seeds and saturated after just 10 training epochs.

These results are systematically decomposed with ablations and oracle experiments, which reveal that matching quality in the representation space is the dominant factor rather than the particular update rule (delta vs. copy).

Figure 2: Per-step dynamics cosine similarity reveals stable alignment between borrowed and actual dynamics directions under oracle matching, but rapid collapse due to drift under regular relay.

Figure 3: Per-step relative $k$3 error across 10-step rollout for all methods, displaying monotonic drift-driven error accumulation.

Figure 4: Wavenumber-resolved spectral relative error at final rollout; U-Net achieves lowest error across all spatial scales, particularly outperforming at low wavenumbers.

Mechanistic Dissection and Diagnostics

Ablations and Oracle Probes: A $k$4 structure explicitly isolates representation geometry (PCA, raw field, ConvAE) from the update rule (delta, copy). Upgrading the matching space yields larger gains than optimizing the update strategy.

Oracle Analyses: When target ground truth is supplied for matching (oracle relay), the relay's error drops from $k$5 to $k$6, exposing autoregressive drift as the chief bottleneck responsible for roughly 12 percentage points of deployable error. Borrowed dynamics directions remain transferable (cosine similarity $k$7), but magnitudes diverge due to intrinsic Re scaling—scaling fixes yield only marginal further gains.

Mechanism Abandonments: Interventions such as adaptive ride length, dynamics-aware autoencoder objectives, and history-matching ablations fail to meaningfully improve performance, confirming the qualitative diagnosis that representation geometry and drift dominate.

Boundary and Regime Shift Analyses

At $k$8, database relay methods fail, with errors ($k$9) exceeding both persistence and FNO, while the U-Net retains a relative advantage ($\mathrm{Re}{\approx}1{,}000$0). This regime boundary is interpreted as a coverage breakdown: the target regime's state space becomes insufficiently indexed by the source-trained database, so representation geometry alone cannot save predictive accuracy. Similarly, increasing database size beyond a few hundred trajectories yields diminishing returns, confirming that representation quality dominates improvement for tractable regime shifts, but support coverage becomes critical for larger extrapolations.

Figure 5: Cross-regime errors show that ConvAE-Relay outperforms FNO at $\mathrm{Re}{\approx}1{,}000$1 Re shift, but this pattern reverses at $\mathrm{Re}{\approx}1{,}000$2, while U-Net consistently dominates.

Figure 6: Relay scaling behavior shows database size ablation (left) and long-horizon error degradation (right) consistent with database coverage and drift effects.

Figure 7: U-Net OOD performance saturates within 10 epochs; further in-distribution improvement does not translate to OOD benefit.

Figure 8: Enstrophy spectrum ratio $\mathrm{Re}{\approx}1{,}000$3 indicates differential damping and amplification patterns by method across spatial scales at rollout step 10.

Theoretical and Practical Implications

Local, Multi-Scale Representations: The confluence of evidence, from both retrieval and forward-prediction modalities, establishes that architectures or representations that preserve local and multi-scale structure yield markedly better transfer across Re shifts. This insight directly challenges the adequacy of global spectral or purely linear representations for robust cross-regime generalisation in PDE systems.

Retrieval vs. Prediction Tradeoff: The empirical failure of dynamics-aware autoencoders to simultaneously optimise for metric indexability and predictive geometry exposes a fundamental tradeoff in learned representations: spaces optimal for retrieval (nearest-neighbour borrowing) do not overlap with those optimal for generation (forward dynamics prediction), a phenomenon likely to generalise to other sequence and operator learning domains.

Engineering Targets: The quantification of drift as the dominant error mode sets a clear engineering goal: anti-drift strategies (e.g., pushforward training, diffusion-based refinement) are poised to deliver substantial error reductions. Temporal context windows and phase-aware encoders represent future directions for breaking the relay coverage boundary and extending database-based transfer.

Methodological Scope: All findings are stringently benchmarked on the forced 2D Navier–Stokes testbed; generalisation beyond this setting remains open but the organising hypothesis is testable on other PDEs and regime shifts.

Conclusion

This study rigorously demonstrates that, for neural and retrieval-based PDE solvers under a substantial Reynolds-number shift, representation geometry is the main driver of generalisation. Local, multi-scale representations such as those induced by skip-connected U-Nets or convolutional autoencoders outperform global spectral or linear representations for both learned forward models and nonparametric retrieval analogues. The work offers a transparent quantification of error sources and sets precise engineering targets for advancing cross-parameter robust PDE solvers. The representation tradeoffs and coverage boundaries identified herein provide fertile ground for further research in both theory and large-scale scientific applications.