Stochastic Asymptotical Regularization (SAR)

• Stochastic Asymptotical Regularization (SAR) is a method that uses stochastic differential equations to address ill-posed inverse problems by adding random perturbations.
• It achieves optimal convergence through bias–variance decomposition and source conditions, applicable to both linear and nonlinear operator equations in Hilbert spaces.
• SAR supports uncertainty quantification via ensemble averaging and computational schemes like Euler–Maruyama and Exponential Euler for robust solution recovery.

Stochastic Asymptotical Regularization (SAR) is a framework for stabilizing and quantifying uncertainty in the approximate solution of ill-posed inverse problems. Rooted in the continuous-time regularization theory, SAR introduces stochastic differential equations (SDEs) as artificial dynamical systems, thereby generalizing classical deterministic asymptotical regularization to random perturbations while enabling rigorous uncertainty quantification. SAR has been systematically developed for both linear and nonlinear operator equations in Hilbert spaces, leading to optimal regularization guarantees and computational schemes with strong mean-square convergence properties (Lu et al., 2020, Zhang et al., 2022, Long et al., 2022).

1. Mathematical Formulation and Theoretical Framework

In the Hilbert space setting, consider the linear inverse problem $y^\delta = A u^\dagger + \delta \eta$, where $A: X \to Y$ is a compact operator, $\eta$ is Gaussian noise with covariance $\Sigma$, and $\delta>0$ the noise level. SAR interprets the solution process as an SDE or stochastic gradient flow whose Itô formulation, for general compact $A$, is

$$d x^\delta(t) = A^* \bigl( y^\delta - A x^\delta(t) \bigr) \, dt + f(t) \, dB_t, \quad x^\delta(0) = x_0.$$

Here, $B_t$ is an $X$-valued $Q$-Wiener process, and $f(t)$ is a deterministic noise-weighting function. The associated mild formulation is

$$x^\delta(t) = e^{-A^*A t} x_0 + \int_0^t e^{-A^*A (t-s)} A^* y^\delta ds + \int_0^t e^{-A^*A (t-s)} f(s) dB_s.$$

The method generalizes to nonlinear operators $F: D(F) \subset X \to Y$, yielding the SDE

$$d x^\delta(t) = F'(x^\delta(t))^* \bigl( y^\delta - F(x^\delta(t)) \bigr) dt + f(t) dB_t.$$

The drift is a stochastic analog of the (continuous) Landweber iteration; the vanishing noise term, governed by $f(t) \downarrow 0$, ensures convergence and optimality (Zhang et al., 2022, Long et al., 2022).

2. Regularization Properties: Convergence, Rates, and Stopping Rules

SAR achieves mean-square convergence of the regularized solution under minimal assumptions on the noise process and weighting: $$\mathbb{E} \| x^\delta(t^*) - x^\dagger \|^2 \to 0 \text{ as } \delta \to 0, \quad t^* \to \infty,\quad \delta t^* \to 0.$$ A bias–variance decomposition leads to sharp bounds under standard source conditions. For source functions $x_0-x^\dagger = \varphi(A^*A)v$, two canonical cases are:

• Hölder source ($\varphi(\lambda) = \lambda^p$): rate $$\mathbb{E}\|x^\delta(t^*)-x^\dagger\|^2 = \mathcal{O}(\delta^{4p/(2p+1)})$$.
• Logarithmic source ($\varphi(\lambda) = \ln^{-\mu}(1/\lambda)$): rate $\mathcal{O}(\ln^{-2\mu}(1/\delta))$.

A priori choice of the terminal time uses the equation $t^* = \Theta^{-1}(\delta)$ where $\Theta(t) = t^{-1/2} \varphi(1/t)$. Stochastic discrepancy principles provide a posteriori stopping: $$t^* = \inf \{ t \geq 0 \;:\; \|A \mathbb{E} x^\delta(t) - y^\delta\|-\tau \delta \leq 0 \} \quad \text{or} \quad \mathbb{E}\|A x^\delta(t)-y^\delta\|^2-\tau \delta^2 \leq 0.$$ Here, $\tau > 1$ balances risk and regularization (Zhang et al., 2022).

Key converse results show the equivalence between the rate of mean-square decay and a spectral-tail condition on the initial guess; this establishes that the range-type source condition is not only sufficient but necessary for order-optimality.

3. Relation to Classical and Modern Regularization Methods

SAR encompasses and extends classical dynamical regularization flows. For $f(t)\equiv 0$ (purely deterministic), the Itô SDE reduces to the Showalter method; time-evolving covariance links SAR to nonstationary asymptotical regularization, while fixed covariance variants connect to 3DVAR. The Kalman–Bucy filter interpretation yields time-continuous variant of Tikhonov regularization, with choice of covariance $C(t)$ providing flexibility for optimal error control (Lu et al., 2020).

The bias–variance structure reveals that nonstationary ARM (Kalman–Bucy) achieves minimax rates under spectral source conditions, provided the tuning parameter $\alpha$ is selected according to the effective dimension. Stationary ARM (3DVAR) exhibits infinite qualification and outperforms in the over-smooth regime, but otherwise lags due to a stagnating variance floor unless appropriately tuned.

4. Numerical Realization and Algorithmic Strategies

Exact simulation in infinite dimensions is infeasible; practical SAR uses strong-order 1 time discretizations:

Scheme	Update Equation	Order
Euler–Maruyama	$$x_{k+1} = x_k + \Delta t\,A^*(y^\delta - A x_k) + f_k \Delta B_k$$	1
Exponential Euler	$$x_{k+1} = e^{-A^*A \Delta t}(x_k + \Delta t\,A^*y^\delta + f_k \Delta B_k)$$	1

Here, $f_k$ is evaluated as $f(t_k)$. In the nonlinear case, $A$ is replaced by the local Fréchet derivative and its adjoint. The discretized $\Delta B_k$ is sampled as independent increments of the Wiener process.

Stopping rules monitor the residual or discrepancy function along each stochastic path and halt when target thresholds are crossed. Ensemble averages over sampled paths yield mean estimates and uncertainty quantification through pointwise empirical variances.

5. Uncertainty Quantification and Inverse Problem Diagnostics

A principal feature of SAR is intrinsic uncertainty quantification: ensembles of stochastic trajectories generate, at stopping time, distributions over possible solutions consistent with data and noise. This facilitates:

• Extraction of pointwise confidence intervals for recovered parameters.
• Detection of multiple solution branches (multimodality) when the inverse problem is nonunique.
• Sampling-based diagnostic tools that identify the presence of hidden structure, e.g., minor binding events in biosensor tomography (Zhang et al., 2022).
• "Optimal path" selection from among the sampled ensemble by criteria such as minimal empirical residual or dominance within a cluster (Long et al., 2022).

These features provide insights that are generally inaccessible to deterministic regularization flows.

6. Extensions: Nonlinear Problems and Practical Advantages

The SAR framework extends to nonlinear inverse problems by using stochastic gradient flows in the space of states. With standard tangential-cone and sourcewise conditions, mean-square convergence and explicit convergence rates remain valid. In particular, SAR enables:

• Escaping from local minima due to stochastic perturbations, which classical Landweber and Tikhonov methods cannot guarantee.
• Discovery of multiple solutions by analyzing sample clusters.
• A generic path to uncertainty quantification in settings where the forward map is nonlinear and the set of solutions is nontrivial (Long et al., 2022).

Empirical comparisons show lower RMSEs versus deterministic Landweber iteration and improved discovery of physically relevant solutions under nontrivial noise and modeling discrepancies.

7. Illustrative Examples and Computational Performance

Application to Fredholm integral equations demonstrates that SAR delivers mean solutions and pointwise confidence bands closely matching known truths even under moderate to high noise. In real-data applications such as SPR biosensor tomography, SAR not only reproduces observable features with high fidelity but also quantifies subdominant binding events undetected by conventional approaches (Zhang et al., 2022).

Computational costs are comparable to deterministic iterative inversions, and SAR propagates uncertainty automatically without requiring explicit prior modeling of noise structure beyond Hilbert-space Wiener processes.

---

Primary sources:

• "On the asymptotical regularization for linear inverse problems in presence of white noise" (Lu et al., 2020)
• "Stochastic asymptotical regularization for linear inverse problems" (Zhang et al., 2022)
• "Stochastic asymptotical regularization for nonlinear ill-posed problems" (Long et al., 2022)