Stochastic Inverse Problem (SIP)

• Stochastic Inverse Problems (SIPs) are inverse problems where the unknown is a probability distribution or random field, used to match observed stochastic data.
• SIPs require rigorous measure-theoretic foundations and regularization methods to address ill-posedness and non-uniqueness inherent in high-dimensional noisy settings.
• Numerical approaches for SIPs include geometric sampling, gradient flow methods, and neural parameterization, enabling uncertainty quantification in diverse scientific applications.

A stochastic inverse problem (SIP) is an inverse problem whose unknown is a probability distribution or random field, rather than a deterministic parameter, and whose data are in the form of stochastic, noisy, or distributional observations. SIPs arise when model parameters are inherently uncertain, data are random, or the modeling process targets population-level or distributional consistency rather than pointwise recovery. The mathematical and computational complexity of SIPs exceeds that of classical deterministic inverse problems, as the solution is a probability measure on the parameter space, determined up to information provided by stochastic observables and geometric properties of the forward map. SIPs span a wide range of applications: from uncertainty quantification in physical models, to statistical learning, stochastic control, system identification in the presence of intrinsic and extrinsic noise, and the discovery of governing equations under data variability.

1. Mathematical Formulations and Measure-Theoretic Foundations

The formal mathematical framework for SIPs is measure-theoretic and geometric. Let Λ ⊂ ℝⁿ denote a parameter space with Borel σ-algebra 𝔹Λ and a reference measure μΛ. A model output or "quantity of interest" (QoI) map Q: Λ → 𝒟 ⊂ ℝᵐ, with typically m ≤ n, relates parameters to data distributions. The SIP is: given a probability measure P_𝒟 on the data domain (often specified by an observed or desired distribution), determine a probability measure P_Λ on parameters such that the push-forward Q♯P_Λ = P_𝒟.

A key feature, particularly when m < n, is the presence of generalized contour maps: the preimage Q⁻¹(d) of data points d ∈ 𝒟 are high-dimensional submanifolds ("contours") in Λ. The paper (Butler et al., 2014) defines a contour σ-algebra 𝒞Λ ⊂ 𝔹Λ generated by these sets, introduces the notion of disintegration of measures, and establishes that a solution to the SIP is a measure on Λ whose marginalization over contours recovers P_𝒟. The Radon–Nikodym derivative dP_Λ/dμ_Λ and properties of measure disintegration underpin existence and uniqueness.

For more general mappings, (Marcy et al., 2022) shows that, under mild assumptions, every SIP solution is fundamentally a change-of-variables: if Q is invertible, f_Λ(λ) = f_𝒟(Q(λ)) * |det DQ(λ)|, and in cases of non-invertibility, mixtures or families of solutions arise due to the non-uniqueness along contour fibers.

2. Regularization, Stability, and Supplemental Information

SIPs are typically ill-posed: different distributions on Λ can yield the same data distribution P_𝒟, especially in the many-to-one (m < n) regime. Well-posedness and stability, therefore, depend on regularization and supplemental information.

Stability is context- and metric-dependent. Using the Wasserstein metric, the stability of the solution with respect to data perturbations is directly linked to the continuity properties of the inverse QoI map: if G⁻¹ is Hölder continuous, then Wₚ(ρΛ*, ρΛ^δ) ≤ C·Wₚ(ρ𝒟*, ρ𝒟^δ)^β (Li et al., 30 Sep 2024). In contrast, when using f-divergences (e.g., Kullback–Leibler), the sensitivity to the forward map is not exacerbated; D_f(ρΛ*, ρΛ^δ) = D_f(ρ𝒟*, ρ𝒟^δ).

To resolve non-uniqueness (underdeterminacy), (Uy et al., 2019) prescribes additional information: (a) known parameter moments, leading to a maximum entropy principle, or (b) parametric family constraints. These reduce the solution set to those distributions on Λ consistent with known statistics or parametric assumptions, regularizing the SIP and facilitating predictions for unobserved QoIs.

An explicit variational regularization framework is developed in (Li et al., 30 Sep 2024): minimize E[ρΛ; ρ𝒟^δ] = D(Q♯ρΛ, ρ𝒟^δ) + R(ρΛ) over probability measures ρΛ, with the loss D and the regularizer R tailored to entropic or Wasserstein metrics. For entropy-entropy regularization, the minimizer is explicit: ρΛ^δ ∝ [(Q⁻¹♯ρ𝒟^δ) * M^α]^{1/(1+α)}.

3. Numerical and Algorithmic Approaches

SIP solution algorithms must address both the high-dimensionality and the stochastic nature of the uncertainty measures. Several foundational strategies have been advanced:

• Geometric Sampling and Non-Intrusive Counting Measures: Exploiting the geometry of contour maps, (Butler et al., 2014) constructs “non-intrusive” Voronoi-based sampling of Λ, assembling empirical probabilities by counting the intersections of parameter samples with induced output space partitions. This approach is resilient to high-dimensionality and is theoretically justified by stochastic geometry; error and convergence analyses separate stochastic (sampling) and deterministic (model evaluation) errors and provide rigorous guarantees for the computation of P_Λ.
• Gradient Flow Methods in Probability Space: For variational SIP formulations, gradient flows with respect to the Wasserstein metric provide an evolution equation for ρΛ on the space of probability measures (Li et al., 30 Sep 2024). The associated PDE, ∂ₜρΛ = ∇·(ρΛ ∇(δE/δρΛ)), admits both Eulerian (density-based) and Lagrangian (particle, ensemble-based) discretization; exponential convergence can be achieved when G is linear and the data distribution is log-concave.
• Statistical Learning and SGD: In statistical inverse problems (interpreted as SIPs), stochastic gradient descent (SGD) is adapted to functional parameter estimation; unbiased stochastic gradients are computed via adjoint operators and smoothed by base learners (Fonseca et al., 2022). Consistency and finite sample excess risk bounds are established.
• Flow Matching and Neural Parameterization: Data-aware flow matching methods—such as DAWN-FM (Ahamed et al., 6 Dec 2024)—learn a neural velocity field between samples from a reference distribution and observed noisy data, allowing for efficient mapping and uncertainty quantification by forward ODE integration.
• Particle Approximations and Bootstrap Sampling: Posterior and push-forward measures are approximated via particle ensembles, especially for high-dimensional or nonparametric settings (Espinosa et al., 5 Sep 2025, Huang et al., 2 Jul 2025, Olabiyi et al., 13 Jul 2025); kernel density estimation and matched-block bootstrap methods support modeling in data-limited and highly stochastic environments.
• Riemannian Optimization for Matrix-Valued SIPs: When the SIP amounts to reconstructing a stochastic matrix with prescribed spectral properties, constrained optimization over manifold structures (including extensions to block-diagonal similarity transformations for complex-conjugate eigenvalues) are solved with Riemannian conjugate gradient techniques (Steidl et al., 2020).

4. Case Studies and Applied Domains

SIPs underpin new methodologies across fields:

• Fluid Flow, Epidemiology, Power Systems: General sample-based approaches (e.g., Voronoi counting, proper scoring rules) are demonstrated on stochastic PDEs in groundwater flow, epidemic modeling, and power grid parameter identification (Butler et al., 2014, Constantinescu et al., 2018). Direct distribution comparison using energy and variogram scores enables robust parameter field estimation and calibrated probabilistic predictions.
• Stochastic Wave and Diffusion Equations: Inverse source problems for stochastic wave equations with driving noise from fractional Brownian motion or finite-jump Lévy processes are addressed via rigorous mild solution theory, spectral decomposition, and regularization (Feng et al., 2021, Huang et al., 2 Jul 2025). Uniqueness is established via expectation and covariance analysis; stability is attained through truncation and multi-frequency data fusion.
• Statistical Learning and Bilevel Optimization: Stochastic bilevel methods, derivative-free when gradients are unavailable, target both model and regularization parameter learning in non-smooth convex variational formulations (Staudigl et al., 27 Nov 2024). Complexity analysis shows bias-error remains controlled under mild conditions.
• Cryo-Electron Microscopy (Cryo-EM): The SIP perspective generalizes discrete conformation models to distributions over molecular structures, with the data as the push-forward through a random forward operator, and utilizes Wasserstein gradient flows over empirical measures to reconstruct the continuous distribution of conformations (Espinosa et al., 5 Sep 2025).
• Physics Discovery under Uncertainty: SIPs are applied to model discovery in systems with input variability and measurement noise, using KL divergence minimization between push-forward and empirical distributions of observables, providing interpretable posteriors over physical laws (Olabiyi et al., 13 Jul 2025).

5. Theoretical Insights and Comparison with Classical Inference

The SIP framework is distinct from Bayesian and classical statistical inference in several ways (Marcy et al., 2022):

• The observable distribution is interpreted as the full population law, not a noisy conditional density; the SIP seeks a parameter distribution whose push-forward exactly (or nearly) matches the data distribution, not merely to condition on sample data.
• SIPs demand p ≥ q (parameters at least as numerous as observables); classical inference often expects more data than unknowns.
• Non-uniqueness is fundamental in underdetermined settings: infinitely many SIP solutions may satisfy the push-forward constraint, parameterized by auxiliary or contour variables. Regularization and supplemental constraints are thus essential for predictive capability.
• Standard statistical model fit and predictive updating (as data increases) do not trivially transfer to SIPs because of the inversion-as-exact-matching paradigm.

These differences emphasize the need for critical analysis regarding where and how the SIP formulation is most appropriate, and caution against uncritical analogy with classical statistical or Bayesian approaches.

6. Challenges, Current Directions, and Open Problems

Major technical and practical challenges remain:

• Efficient high-dimensional sampling and integration schemes, especially for complex, multi-modal, or implicitly defined contour structures.
• Regularization strategies responsive to the SIP’s unique stability and identifiability characteristics—balancing bias and variance in probability measures rather than pointwise parameter estimates.
• Quantifying and propagating uncertainty through nonlinear, possibly non-invertible forward maps, particularly for systems with partial observability and non-Gaussian noise.
• Structure-exploiting algorithms for domains such as space-semidiscrete stochastic PDEs (Lecaros et al., 3 Sep 2025), inverse control via stochastic maximum principles (Nakano, 2020), and stochastic inverse eigenvalue problems for non-symmetric matrices (Steidl et al., 2020).
• Theoretical clarification on the limits of the SIP framework, including conditions for uniqueness and principled selection among non-unique solutions (Marcy et al., 2022).
• Application to data-driven model discovery, where system variability and uncertainty preclude deterministic coefficient estimation (Olabiyi et al., 13 Jul 2025).

7. Significance and Impact

SIPs formalize and generalize the quantification of uncertainty in inverse problems, moving from estimates of individual parameter values to population-level descriptions, and from pointwise inversion to geometric and statistical matching. The SIP paradigm has advanced robust solution methodologies in settings characterized by high-dimensional, noisy, incomplete, and distributional data, as found in computational physics, system identification, imaging, and scientific machine learning. Ongoing research continues to refine both the theoretical underpinnings (measure-disintegration, optimal transport, variational regularization) and the computational algorithms (ensemble methods, stochastic optimization, deep generative models) necessary for SIPs to address contemporary scientific and engineering challenges.