PDE-Constrained Generation

• PDE-Constrained Generation is a framework for generating probability measures over parameters while strictly enforcing PDE constraints and matching observational data.
• It employs particle-based approximations, adjoint PDE solvers, and deep generative models to compute gradient flows and balance soft and hard constraint enforcement.
• This approach enables robust uncertainty quantification, inverse design, and model calibration in complex systems such as subsurface flow, elasticity, and wave propagation.

Partial differential equation (PDE)-constrained generation is the probabilistic or generative modeling of parameter fields, control inputs, or solution states in systems governed by PDEs, where the generation process is subject to PDE constraints and potentially additional observations, uncertainty quantification, or distribution-matching objectives. This paradigm generalizes classical PDE-constrained optimization to settings where the goal is to generate or sample distributions of states or parameters consistent with both physical laws and data, often leveraging advanced tools from optimization, machine learning, and optimal transport. Modern approaches address both deterministic and stochastic parameter inference, complex priors, high-dimensional and nonlinear PDEs, and strict or soft satisfaction of governing equations.

1. Mathematical Formulation of PDE-Constrained Generation

Let $\Theta\subset\mathbb{R}^m$ be a parameter space, and $\mathcal{Y}\subset\mathbb{R}^n$ the space of observables. The PDE-constrained generation problem typically seeks a probability measure $\mu$ on parameters $\theta$ such that the push-forward measure $\nu = S_\#\mu$ (where $S:\Theta\to\mathcal{Y}$ is the solution operator, implicitly defined by $g(u,\theta)=0$ or $\mathcal{L}(u;\theta)=f$) matches an observed or target data law $\nu_{\text{obs}}$ according to a divergence $D(\nu,\nu_{\text{obs}})$ (e.g., Kullback–Leibler, Wasserstein-2) (Li et al., 2023).

The canonical variational problem is: $\mathcal{Y}\subset\mathbb{R}^n$0 This framework extends classical deterministic inverse problems—where $\mathcal{Y}\subset\mathbb{R}^n$1 is optimized to fit data under the PDE constraint—to measure-valued settings, capable of capturing stochasticity or distributional properties of the unknowns.

The associated gradient flow in the Wasserstein metric on the space of probability measures $\mathcal{Y}\subset\mathbb{R}^n$2 is governed by the continuity equation: $\mathcal{Y}\subset\mathbb{R}^n$3 Where the variational derivative is given by: $\mathcal{Y}\subset\mathbb{R}^n$4 with $\mathcal{Y}\subset\mathbb{R}^n$5 the data-space potential (e.g., $\mathcal{Y}\subset\mathbb{R}^n$6 for KL divergence) (Li et al., 2023).

2. Variational Principles and Particle-Based Algorithms

To operationalize gradient flows over distributions, empirical approximations via finite ensembles (particles) are standard: $\mathcal{Y}\subset\mathbb{R}^n$7 At each iteration:

1. Solve forward PDE for each $\mathcal{Y}\subset\mathbb{R}^n$8 to get $\mathcal{Y}\subset\mathbb{R}^n$9.
2. Estimate the data-space potential $\mu$0 from $\mu$1 and the target law.
3. Compute gradients with respect to $\mu$2, either by differentiating $\mu$3 explicitly or solving an adjoint PDE: for loss functions $\mu$4 that depend on $\mu$5, adjoint-based methods solve $\mu$6.
4. Update $\mu$7 in the negative direction of the gradient flow.

This approach supports uncertainty quantification, model calibration, and inverse design in complex models, subsurface flow, elasticity, and wave propagation (Li et al., 2023). Ensemble sizes commonly range from $\mu$8 to $\mu$9, and kernel density estimation in data space may be required for likelihood ratio estimation.

3. Integration with Generative Models and Flow-Based Frameworks

Recent advances extend PDE-constrained generation to the domain of deep generative models. Generative neural reparameterization (GNR) replaces direct parameter optimization with a neural generator $\theta$0 mapping latent noise $\theta$1 to parameters $\theta$2; the expected loss across generated samples is minimized using automatic differentiation through both the generator and the differentiable PDE solver (Joglekar, 2024).

Training samples from $\theta$3, computes $\theta$4 via PDE solves, and updates $\theta$5 using Adam or other optimizers. This enables learning explicit samplers for high-quality parameter distributions, capturing multiple isolated minima, critical for robust design in non-convex landscapes.

Furthermore, score-based generative models—especially diffusion models—enable high-dimensional parameter estimation under complex, non-Gaussian priors. These methods leverage SDE/ODE-based sampling guided by learned score networks, and incorporate physics-informed surrogates or explicit residual constraints to steer generated samples toward physical admissibility and data consistency (Hong et al., 2024).

4. Approaches to Hard and Soft Constraint Enforcement

Constraint satisfaction is a major challenge in PDE-constrained generation. Methods are distinguished by whether constraints are enforced in expectation ("soft") or exactly ("hard"):

• Soft constraint enforcement: Imposes penalization via residual losses in the objective (e.g., mean squared PDE residuals or weak-form errors). Many score-based diffusion models, flow-matching, or operator learning methods rely on this paradigm, adjusting sample trajectories or generator weights to drive the constraint loss toward zero, while balancing generative fidelity (Tauberschmidt et al., 5 Aug 2025).
• Hard constraint enforcement: Develops projection-type algorithms to ensure strict satisfaction of boundary conditions, conservation laws, or manifold restrictions—often zero-shot, i.e., with no retraining. Physics-Constrained Flow Matching (PCFM) computes a correction to the generative flow by projecting the drift onto the tangent space of the constraint manifold, using Lagrange multipliers to solve

$\theta$6

Each step in the flow integration is followed by Gauss–Newton (or analogous) projection to the constraint set, guaranteeing $\theta$7 up to numerical precision (Utkarsh et al., 4 Jun 2025).

Gradient-free approaches such as ECI sampling alternate between extrapolation, closed-form correction, and interpolation in functional flow-matching models, achieving exact hard constraint enforcement in a zero-shot, non-gradient setting by repeatedly projecting onto constraint manifolds and interpolating via optimal transport paths (Cheng et al., 2024).

5. Stochasticity, Uncertainty, and Complex Priors

Many physical inverse problems involve inherently stochastic or high-dimensional unknowns, motivating the formulation of PDE-constrained generation in the space of measures. The framework (Li et al., 2023) generalizes to stochastic settings by seeking a law $\theta$8 on parameters so that the induced data law $\theta$9 matches empirical data distributions under a divergence $\nu = S_\#\mu$0 (e.g., $\nu = S_\#\mu$1, KL), enabling recovery of full distributions of random parameters.

Score-based diffusion models incorporate complex, possibly sample-based priors, using denoising score-matching to approximate $\nu = S_\#\mu$2, and posterior sampling via Stochastic Differential Equations driven by both learned prior scores and data likelihood gradients. A physics-informed neural surrogate enables rapid evaluation of forward models necessary for gradient computation, supporting high-dimensional problems such as hyperelasticity and multiscale mechanics (Hong et al., 2024). This hybridization supports posterior estimation for complex, geometric, or multimodal priors inaccessible to traditional MCMC or variational methods.

6. Applications and Numerical Implementation

Table: Selected Algorithmic Frameworks in PDE-Constrained Generation

Method	Constraint Satisfaction	Principal Components
Particle-based measure gradient flow (Li et al., 2023)	Loss minimization via divergence (soft)	Empirical measures, adjoint PDE, KDE
Generative neural reparameterization (Joglekar, 2024)	Sampled loss minimization (soft)	Latent neural generator, auto-diff PDE
Physics-constrained flow matching (PCFM) (Utkarsh et al., 4 Jun 2025)	Exact manifold projection (hard)	Lagrange multipliers, flow corrections
Gradient-free ECI sampling (Cheng et al., 2024)	Closed-form at every iteration (hard)	Extrapolation, projection, interpolation
Score-based/posterior diffusion (Hong et al., 2024)	Posterior SDE guidance (soft/strong)	Score networks, physics-informed surrogates

Key applications include:

• Uncertainty quantification and model calibration in subsurface flow, elasticity, and wave propagation.
• Inverse design and discovery—finding distributions over microstructure or control parameters yielding desired macroscopic responses or system behaviors.
• Generative modeling for physics-informed data synthesis, robust design, and simulation-augmented analysis.

Computational considerations include:

• The use of differentiable surrogates (DeepONet, PINN, CNN) to replace expensive forward PDE solves (Wang et al., 2021, Hong et al., 2024).
• Efficient adjoint and hypergradient methods (Broyden’s, implicit differentiation) in bi-level and large-scale optimization (Hao et al., 2022).
• Ensemble and particle-based parallelism for scalable uncertainty estimation and distribution matching.

7. Challenges, Limitations, and Outlook

While PDE-constrained generation unifies optimization, generative modeling, and probabilistic inference for physical systems, challenges remain:

• Exact constraint satisfaction in generative models can trade off against sample diversity or fidelity, necessitating careful design of projection, penalty, or reward structures (Zhang et al., 28 May 2025).
• High-dimensional, stiff, or multi-scale PDEs remain challenging for both data-driven and physics-informed models; spectral bias and ill-conditioned losses are persistent issues (Wang et al., 2021).
• Theoretical guarantees of convergence, coverage, and representation fidelity in stochastic, non-convex, or high-dimensional regimes remain underdeveloped for many neural and diffusion-based methods.

Ongoing research explores tighter integration of physics, data, and deep generative methodologies, including post-hoc distillation to mitigate mismatch between expectation-based and sample-based constraint enforcement (Zhang et al., 28 May 2025), adaptive architectures, and uncertainty-aware surrogate modeling. Extensions to coupled multi-physics systems, nonlocal or fractional PDEs, and sensor/data-driven scenario optimization remain active areas of investigation.