Bayesian Inversion for Stochastic Pinning

• Bayesian inversion for stochastic pinning is a probabilistic framework that infers spatially random pinning fields in PDEs using noisy observations and prior information.
• It employs parameterizations like the Karhunen–Loève expansion along with physics-informed neural networks to accurately model and predict pinning effects.
• Advanced methods such as HMC, VI, SSLE, and replica-exchange MCMC ensure robust uncertainty quantification, efficient posterior exploration, and accurate parameter inference.

Bayesian inversion for stochastic pinning refers to a class of inverse problem methodologies in which one seeks to infer parameters or fields associated with "pinning" effects (e.g., spatially random forces, quenched disorder in material systems) from noisy observations, under a probabilistic framework that quantifies uncertainty and leverages prior knowledge. The stochastic nature of the pinning field introduces intrinsic randomness, demanding specialized parameterizations (such as Karhunen–Loève expansions), integration with physics-informed surrogates (e.g., PINNs), and rigorous posterior inference algorithms including both sampling-based and surrogate-based techniques. Multiple frameworks have been developed for addressing such inverse problems, notably Bayesian physics-informed neural networks (B-PINNs) (Yang et al., 2020), stochastic spectral likelihood embedding (SSLE) (Wagner et al., 2020), multi-variance replica-exchange SGLD for non-convex posteriors (Lin et al., 2021), and measure-consistent Bayesian push-forward approaches (Butler et al., 2017).

1. Bayesian Formulation for Stochastic Pinning Models

Stochastic pinning problems are governed by PDEs with unknown (and random) spatially varying fields $a(x)$ acting as "pinning" terms. The canonical forward model is

$$N_x[u(x); a(\cdot)] = 0, \quad x \in D \subset \mathbb{R}^d,$$

with prescribed boundary conditions. A prototypical example is the steady-state equation,

$$-\partial_x(\kappa \partial_x u) + V'(u) + a(x) = 0, \quad x \in [0,1],$$

where $a(x)$ models the local pinning effect and is to be inferred from sparse and noisy measurements of the field $u(x)$.

The Bayesian inverse problem is to infer the posterior distribution on $a(x)$ (or its parameterization) conditioned on observed data $\{(x_i, y_i)\}_{i=1}^N$, where $y_i = u(x_i) + \epsilon_i$, $\epsilon_i \sim \mathcal{N}(0, \sigma^2)$. The full Bayesian formulation specifies a joint prior on the unknowns, a likelihood for the data, and yields a posterior via Bayes’ rule.

2. Field Parameterization: Karhunen–Loève Expansion and Priors

A key step is representing the unknown random field $a(x)$ in a tractable, finite-dimensional form. A standard approach is to use a truncated Karhunen–Loève (KL) expansion:

$$N_x[u(x); a(\cdot)] = 0, \quad x \in D \subset \mathbb{R}^d,$$0

where $$N_x[u(x); a(\cdot)] = 0, \quad x \in D \subset \mathbb{R}^d,$$1 are leading eigenpairs of the prior covariance operator of $$N_x[u(x); a(\cdot)] = 0, \quad x \in D \subset \mathbb{R}^d,$$2 and $$N_x[u(x); a(\cdot)] = 0, \quad x \in D \subset \mathbb{R}^d,$$3. The Bayesian prior is then $$N_x[u(x); a(\cdot)] = 0, \quad x \in D \subset \mathbb{R}^d,$$4, and the overall parameter space is augmented to jointly include the coefficients of the physical surrogate (e.g., neural network weights) and the KL coefficients $$N_x[u(x); a(\cdot)] = 0, \quad x \in D \subset \mathbb{R}^d,$$5.

In the B-PINN setting, the prior on the total parameter vector $$N_x[u(x); a(\cdot)] = 0, \quad x \in D \subset \mathbb{R}^d,$$6 is taken as a product of independent Gaussians for neural network weights and KL coefficients:

$$N_x[u(x); a(\cdot)] = 0, \quad x \in D \subset \mathbb{R}^d,$$7

(Yang et al., 2020).

3. Bayesian Physics-Informed Neural Networks and Posterior Inference

B-PINN Construction

A Bayesian neural network surrogate $$N_x[u(x); a(\cdot)] = 0, \quad x \in D \subset \mathbb{R}^d,$$8 is trained to approximate the latent physical field, with the loss functional comprising data misfit and physics residual:

• Data misfit: $$N_x[u(x); a(\cdot)] = 0, \quad x \in D \subset \mathbb{R}^d,$$9
• Physics residual (PINN loss): $$-\partial_x(\kappa \partial_x u) + V'(u) + a(x) = 0, \quad x \in [0,1],$$0

The negative log-posterior (up to constants) is

$$-\partial_x(\kappa \partial_x u) + V'(u) + a(x) = 0, \quad x \in [0,1],$$1

(Yang et al., 2020).

Posterior Estimation

The posterior $$-\partial_x(\kappa \partial_x u) + V'(u) + a(x) = 0, \quad x \in [0,1],$$2 is approximated via:

• Hamiltonian Monte Carlo (HMC): Produces asymptotically exact samples of the posterior but requires multiple full gradient evaluations per sample (computationally intensive as dimensionality increases).
• Variational Inference (VI): Uses a mean-field (product Gaussian) variational family $$-\partial_x(\kappa \partial_x u) + V'(u) + a(x) = 0, \quad x \in [0,1],$$3 and maximizes the evidence lower bound (ELBO); faster and scalable but with known mode-seeking behavior and underestimation of uncertainty.

Comparison studies indicate that HMC captures posterior uncertainty more robustly than VI, but at higher computational cost. Both approaches enable quantification of uncertainty in the recovered physical field and the pinning map (Yang et al., 2020).

4. Surrogate-Based and Sampling-Free Inversion Approaches

Stochastic spectral likelihood embedding (SSLE) provides a sampling-free surrogate-based inversion framework (Wagner et al., 2020). SSLE represents the likelihood function $$-\partial_x(\kappa \partial_x u) + V'(u) + a(x) = 0, \quad x \in [0,1],$$4 (where $$-\partial_x(\kappa \partial_x u) + V'(u) + a(x) = 0, \quad x \in [0,1],$$5 encodes all unknowns, including pinning coefficients) as a piecewise polynomial chaos expansion (PCE) constructed adaptively through the stochastic spectral embedding (SSE) methodology. Terminal subdomains of parameter space are refined based on cross-validation and variance-gap error indicators.

The SSLE surrogate enables closed-form computation of posterior moments, evidence, and marginals by algebraic manipulation of the PCE coefficients. Adaptive sample enrichment (adSSLE) targets refinement in regions with informative likelihood, accelerating convergence. For stochastic pinning models—where the forward map may introduce thresholds or multimodal landscapes—adSSLE discovers regions of high posterior mass, efficiently approximating the posterior even in high dimensions, provided the effective dimensionality is moderate (Wagner et al., 2020).

5. Advanced Sampling: Replica-Exchange and Multi-Fidelity MCMC

Multi-variance replica-exchange stochastic gradient Langevin diffusion (m-reSGLD) addresses the challenge of multi-modal and non-convex posteriors characteristic of nonlinear stochastic pinning PDEs (Lin et al., 2021). The method simulates parallel stochastic Langevin chains at different "temperatures" (noise levels) and allows for state-exchange moves with an unbiased swap-rate estimator. The algorithm supports the use of solvers of differing fidelity (fine for low-T, coarse for high-T) and corrects for swap-rate bias due to stochastic energy estimates.

Empirical results indicate that m-reSGLD accelerates convergence, efficiently explores multimodal posteriors, and yields robust parameter inference for Bayesian PINNs arising in inverse and forward stochastic PDEs. The technique mitigates trapping in local modes—a key difficulty in conventional MCMC when applied to non-convex inversion problems (Lin et al., 2021).

6. Consistent Bayesian Push-Forward Approaches for Stochastic Parameters

Consistent Bayesian inversion constructs a posterior on parameter space whose push-forward via the forward map exactly reproduces the observed probability law on the data (Butler et al., 2017). For stochastic pinning, where the outputs arise from both model parameters (e.g., stiffness) and intrinsic randomness (e.g., pinning angles), the framework extends by treating parameters and random inputs jointly. The consistent posterior is:

$$-\partial_x(\kappa \partial_x u) + V'(u) + a(x) = 0, \quad x \in [0,1],$$6

where $$-\partial_x(\kappa \partial_x u) + V'(u) + a(x) = 0, \quad x \in [0,1],$$7 is the observable, $$-\partial_x(\kappa \partial_x u) + V'(u) + a(x) = 0, \quad x \in [0,1],$$8 is the known density of the random pinning input, and $$-\partial_x(\kappa \partial_x u) + V'(u) + a(x) = 0, \quad x \in [0,1],$$9 is the push-forward of the joint prior on observables.

This approach guarantees that the push-forward of the posterior matches the observed probability measure, maintains the marginal law on known random variables (i.e., pinning angle prior remains unchanged), and enables inference for the principal unknowns (e.g., stiffness) via Monte Carlo sampling and kernel density estimation (Butler et al., 2017).

7. Uncertainty Quantification and Trade-Offs

Bayesian inversion for stochastic pinning enables comprehensive uncertainty quantification:

• B-PINN/HMC: Posterior draws provide empirical means and variances for both $a(x)$0 and $a(x)$1, robustly reflecting model and data uncertainty, but incur high computational cost in the presence of high-dimensional KL expansions (Yang et al., 2020).
• SSLE/adSSLE: Delivers analytical posterior summaries and is efficient provided the surrogate accurately covers the effective posterior support; effective in both multimodal and high-concentration regimes (Wagner et al., 2020).
• Replica Exchange MCMC: Surmounts multi-modality and slow mixing in difficult loss landscapes, with rigorous error bounds for discretization and sampling (Lin et al., 2021).
• Consistent Bayesian: Uniquely enforces push-forward consistency, with error analysis depending mainly on observable dimension and the quality of density estimation, not the latent parameter dimension (Butler et al., 2017).

Trade-offs involve computational cost (HMC vs. VI vs. surrogate), flexibility in representing field uncertainty (KL truncation limitations), and exactness of uncertainty quantification. For high-dimensional pinning fields, BNN-based surrogates offer greater scalability than finite-mode KL expansion. SSLE is sampling-free but can be limited by surrogate accuracy at sharp or thresholded likelihood structures. Consistent Bayesian methods excel when observable statistics are paramount, while stochastic-gradient and replica-exchange procedures ensure sampling correctness in highly non-convex settings.

---

References

• "B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data" (Yang et al., 2020)
• "Bayesian model inversion using stochastic spectral embedding" (Wagner et al., 2020)
• "Multi-variance replica exchange stochastic gradient MCMC for inverse and forward Bayesian physics-informed neural network" (Lin et al., 2021)
• "A Consistent Bayesian Formulation for Stochastic Inverse Problems Based on Push-forward Measures" (Butler et al., 2017)