Posterior-First Neural PDE Simulation: Inferring Hidden Problem State from a Single Field

Abstract: Neural PDE simulators often receive only a single observed field at deployment. In this setting, a field-to-future predictor can collapse distinct latent problem states into the same deterministic interface, losing the ambiguity needed for reliable rollout and downstream decisions. We propose posterior-first neural PDE simulation: first infer a posterior over the minimal task-sufficient problem state, then condition prediction on that posterior. The resulting theory connects the object, the learning target, and the failure mode: Bayes downstream values factor through this posterior, refinement labels make it learnable by proper scoring rules, and deterministic collapse incurs an ambiguity barrier whenever the true posterior is non-Dirac. Synthetic exact-ambiguity experiments show that point-versus-posterior gaps track the predicted barrier. On metadata-hidden PDEBench tasks, posterior recovery reduces pooled rollout nRMSE from 0.175 to 0.132, closing 59.4% of the direct-to-oracle gap. These results suggest that single-observation neural PDE simulation should be posterior-first rather than monolithic field-to-future prediction.

• The paper demonstrates the importance of inferring the posterior over minimal task-sufficient hidden states from a single observed field.
• It introduces a two-stage architecture that recovers a full joint distribution over latent variables to enhance downstream predictions.
• Empirical results on synthetic benchmarks and PDEBench show significant error reductions and improved uncertainty quantification.

Posterior-First Neural PDE Simulation: Inferring Hidden Problem State from a Single Field

Problem Statement and Motivation

Neural PDE simulators provide a data-driven alternative for solving and forecasting scientific and engineering systems governed by partial differential equations. Most deployed scenarios supply only a single observed field (e.g., a spatial state at one time), yet expect the simulator to predict future states or answer complex downstream queries. This single-observation regime is inherently ambiguous: key components of the underlying problem state—hidden field variables, coefficients, operators, boundary conditions, or memory—remain unidentified, resulting in multimodal uncertainty over plausible problem physics.

Standard approaches bypass this ambiguity by either embedding inductive biases in the model architecture, supplementing with rich context or extra side-information at test time, or simply collapsing possible latent states to a deterministic point estimate. Such monolithic field-to-future predictors, by failing to explicitly represent unresolved ambiguity, produce collapsed predictive distributions and exhibit systematic utility loss in downstream tasks when the observation does not uniquely identify the latent state.

Formalization: Minimal Task-Sufficient Posterior

The work provides a mathematically rigorous formulation identifying the minimal reusable interface for single-observation neural PDE simulators as the posterior over the minimal task-sufficient problem state. Specifically, for each observation $x_t$ generated via $X_t = \mathcal{O}(W_t)$ from a latent world $W_t$, and for any set of downstream task targets $Y^\tau_t$, the notion of task-sufficiency is formalized as the finest equivalence class $Z^*_t = g^*(W_t)$ such that, conditioned on $Z^*_t$, the joint law of any $Y^\tau_t$ is invariant to the remaining latent details. In practice, $Z^*_t$ is not recoverable from $x_t$; rather, the true Bayesian object available is its conditional posterior $\pi_t(\cdot \mid x_t)$.

This factorization ($X_t = \mathcal{O}(W_t)$0) is demonstrated to be canonical in the sense that: (1) all downstream Bayes-optimal predictions and decisions must pass through this posterior; (2) proper scoring rules (e.g., log or Brier loss) yield exact and unique learning targets for such posteriors; and (3) deterministic point-latent models are provably separated from Bayes-optimal posteriors by an ambiguity barrier whenever the true posterior is non-degenerate.

Theory: The Irreducible Ambiguity Barrier

The authors provide sharp theoretical results. Given a suitable loss for structure recovery (e.g., log or Brier), the excess risk for any predicted distribution $X_t = \mathcal{O}(W_t)$1 over a semantic refinement $X_t = \mathcal{O}(W_t)$2 above the Bayes risk is exactly the expected $X_t = \mathcal{O}(W_t)$3 (log loss) or $X_t = \mathcal{O}(W_t)$4 (Brier), where $X_t = \mathcal{O}(W_t)$5 is the true conditional refinement posterior. Crucially, any deterministic predictor (point latent) incurs a strictly positive irreducible gap relative to the Bayes risk when the conditional $X_t = \mathcal{O}(W_t)$6 is non-Dirac. The scale of this gap increases with the ambiguity induced by the observation and completely disappears only when the ambiguity is negligible.

Furthermore, downstream utility for any decision or surrogate task is Lipschitz in the posterior error, establishing that posterior quality—not backbone size or auxiliary losses—controls simulator performance in the ambiguous regime.

Methodology: Minimal Posterior-First Simulator

The principle is instantiated via a minimal two-stage architecture: first, a posterior encoder produces a full joint distribution $X_t = \mathcal{O}(W_t)$7 over a countable semantic refinement of the problem state; second, this posterior is exposed to a standard neural PDE rollout backbone (e.g., FNO, UNet, ConvLSTM, Transformer) via a learned low-dimensional embedding summary $X_t = \mathcal{O}(W_t)$8, appended to the observed field at each time step. Training optimizes a weighted sum of proper-scoring rule loss for posterior recovery and the usual task or rollout loss used by direct baselines. The only architectural change from a conventional simulator is the requirement to expose and supervise the latent posterior, without increased backbone capacity, hidden-state size, or context windows.

Empirical Results

Synthetic Exact-Ambiguity Benchmark

A synthetic PDE benchmark is constructed where latent ambiguity can be explicitly controlled. The ambiguity barrier—the minimal excess Brier risk for any point-latent model—is precisely characterized. Under controlled ambiguity, the observed gap between point-latent and posterior models matches the predicted theoretical barrier with high precision (ratio $X_t = \mathcal{O}(W_t)$9--$W_t$0 across regimes).

Downstream, the posterior-first simulator closes the majority of the direct-to-oracle rollout gap. In the highest-ambiguity regime, the posterior model reduces normalized gap from $W_t$1 to $W_t$2 (i.e., a strong proportion of the theoretically achievable improvement), with absolute improvements in future NLL, operator identification, and coefficient estimation. Capacity-matched direct models, auxiliary supervision, or output uncertainty provide only partial gains, none of which can exceed the ambiguity limit of point estimates.

Public PDEBench Metadata-Hidden Benchmark

On public PDEBench benchmarks (DR, DS, SW, INS) with realistic PDEs and metadata-hidden evaluation protocols, the posterior-first pipeline yields consistent reductions in nRMSE across all problem families and neural simulator backbones (e.g., reducing pooled $W_t$3 from $W_t$4 to $W_t$5), closing $W_t$6 of the direct-to-oracle gap. Stratifying results by a held-out ambiguity score, the gap closure increases monotonically with ambiguity, peaking at $W_t$7 in the highest ambiguity bin.

Explicit recovery of hidden metadata is verified: for both discrete (classification accuracy) and continuous ($W_t$8) latent variables, the posterior-first model consistently approaches the oracle upper bound and vastly exceeds direct and probabilistic direct alternatives (e.g., $W_t$9 vs $Y^\tau_t$0 mean probe score).

Comparison with uncertainty-aware direct models and probabilistic neural operator baselines confirms that gains are not due to generic predictive uncertainty, but rather to principled recovery and conditioning on task-relevant latent structure.

Implications and Future Directions

This work establishes that, in single-observation neural PDE simulation, optimality and utility hinge on first inferring and making explicit the posterior over hidden problem-state variables, not merely enhancing rollout capacity or auxiliary point supervision. The minimal sufficient statistic for downstream scientific use is the posterior over the minimal task-sufficient hidden state, not a deterministic or unstructured latent.

This interface-centric criterion fundamentally refines how neural PDE surrogates should be constructed, evaluated, and compared. Theoretical findings make clear that average rollout errors or Bayesian-inspired outputs are inadequate metrics for model selection in the ambiguous regime; posterior recovery of relevant problem physics must be assessed directly.

Future research should extend posterior recovery beyond countable and supervised latent refinements, developing continuous, flow-based, or weakly-supervised posteriors compatible with more complex scientific PDEs. Addressing posterior conditioning for continuous variables, combinatorial latent spaces, or multi-modal ground truth will require advances in amortized inference and structured variational methods. Additionally, stability analysis under realistic task families and exploration of computational costs in high-dimensional settings will be essential for practical deployment.

Conclusion

The posterior-first paradigm for neural PDE simulation is both formally justified and empirically validated as the optimal mechanism for exploiting single-field observations in the presence of latent ambiguity. By mandating explicit inference of the posterior over task-relevant hidden states before rollout, this approach sharply improves downstream prediction, surrogate reliability, and scientific interpretability in scenarios where full identification is impossible. The results reframe both the modeling and evaluation standards for neural scientific simulators, establishing a new baseline for principled uncertainty quantification and decision support in scientific ML.