Regularized Inversion Algorithm

• Regularized inversion algorithm is a computational scheme that reconstructs unknown parameters from noisy, ill-posed measurements by integrating prior knowledge and stability constraints.
• It employs techniques such as Tikhonov, total variation, and sparsity regularization to mitigate issues like noise, non-uniqueness, and computational instability.
• The framework combines mathematical formulations, iterative solvers, and adaptive parameter selection, making it applicable across imaging, geophysics, and data-driven applications.

A regularized inversion algorithm refers to any computational scheme that reconstructs an unknown object or parameter from indirect, noisy, and typically ill-posed measurements by incorporating explicit or implicit regularization. Regularization is essential to stabilize inversions by encoding prior information or desired properties (e.g., smoothness, sparsity, physical constraints), as the underlying operator often exhibits ill-conditioning, non-uniqueness, or instability to noise. The concept originated in linear inverse problems but now encompasses a broad class of methods spanning linear and nonlinear operators, probabilistic or deterministic regularization, and optimization-based or iterative solvers. This article outlines foundational mathematical formulations, algorithmic strategies, and major advances in the design and analysis of regularized inversion algorithms.

1. Fundamental Mathematical Formulations

Regularized inversion formalizes parameter recovery as a constrained or penalized optimization problem. The generic linear formulation is: $\min_{x} \; \|A x - b\|_2^2 + \lambda R(x)$ where:

• $A \in \mathbb{R}^{m \times n}$ is a forward or measurement operator (often ill-conditioned or compact),
• $b \in \mathbb{R}^m$ represents noisy observations,
• $R(x)$ is a regularization or stabilizing functional (e.g., Tikhonov $\|Lx\|_2^2$, TV $\|\nabla x\|_1$, sparsity penalties),
• $\lambda > 0$ is the regularization parameter controlling the trade-off between fidelity and prior.

In nonlinear settings, the core structure persists: the inverse problem is

$\min_{x} \; D(F(x), y) + \lambda R(x)$

where $F$ is a (possibly nonlinear) forward map, $y$ is observed data, and $D(\cdot, \cdot)$ encodes the data term (often least-squares or Poisson likelihood).

Variants include constrained (e.g., $\|F(x) - y\|^2 \leq \delta$), multi-term, or bilevel principles (as in PDE-constrained learning (Nguyen, 2024)). For problems involving specific physics (e.g., inverse Radon transform, Kohn-Sham DFT, seismic inversion), the formulation can involve functional spaces and custom regularization reflecting domain properties (Anikin et al., 2024, Herbst et al., 2024).

2. Classical and Modern Regularization Methods

Tikhonov and Smoothness Regularization

Tikhonov regularization uses an analytic (often quadratic) penalty: $\min_x \|A x - b\|_2^2 + \lambda^2 \|L x\|_2^2$ where $L$ is typically the identity (zeroth order), a finite-difference matrix (first/second order smoothness), or more general operators (Hannah et al., 2012). The analytical solution is

$x_\lambda = (A^T A + \lambda^2 L^T L)^{-1} A^T b$

or, in the presence of analytic constraints, using generalized singular value decompositions (GSVD).

Extensions involve incorporating additional analytic regularization operators to target specific spectral or edge features, such as in truncated-angle tomography where an additional quadratic term bolsters robustness to missing data and loss of invertibility (Anikin et al., 2024).

Non-quadratic, Sparsity, and Total Variation Regularization

For piecewise-constant, sparse, or edge-preserving inversion (e.g., geophysics, image restoration),

$R(x) = \|W x\|_1 \ \text{or} \ \mathrm{TV}(x)$

are employed, leading to convex but nonsmooth optimization problems. Iteratively Reweighted Least Squares (IRLS) and projections onto convex sets enable efficient numerical minimization for large systems (Vatankhah et al., 2017, Ito et al., 2019).

Probabilistic, Noise-adapted, and Data-driven Regularization

Maximum-likelihood inversion under non-Gaussian noise motivates specialized data terms (e.g., Poisson log-likelihood for x-ray data (Swanson et al., 2017)): $$L(f) = \sum_j \left[ (A f)_j + b_j - n_j \ln[(A f)_j + b_j] \right]$$ Regularization may take the form of smoothness on physically meaningful transformed variables (e.g., nearest-neighbor energy densities in EEDF recovery) and is selected empirically or via discrepancy principles to balance physical realism and nonnegativity.

Plug-and-play (PnP) regularization further generalizes the regularizer to be any denoiser or learned mapping used as a proximal operator within a primal-dual algorithm, offering superior empirical performance and flexibility to encode sophisticated (often nonlocal and data-driven) priors (Luiken et al., 2024).

3. Algorithmic Strategies and Solution Methodologies

Direct, Analytical Solvers

Where the objective is convex quadratic, closed-form or GSVD-based inversion using Cholesky or SVD is possible. For large-scale systems, classical SVD is replaced by randomized SVD (RSVD) to dramatically reduce computational cost while preserving dominant spectral information and theoretical error control (Ito et al., 2019).

Iterative and Proximal Splitting Methods

Nonsmooth or large-scale problems necessitate iterative schemes:

• Landweber iteration and its superiorized variants, where a base gradient descent is perturbed in descent directions of the regularizer to reduce its value over time without sacrificing convergence to a right-inverse (Gibali et al., 2023).
• Proximal Newton frameworks for nonlinear and composite objectives. The quadratic data term is locally approximated at each iteration, and the nonsmooth regularizer is handled by a proximal operator (possibly a black-box denoiser), with Newton or ADMM-based splitting for efficiency (Aghamiry et al., 2020).
• Iteratively regularized Gauss-Newton (IRGNM) and Levenberg-Marquardt schemes for nonlinear forward models, with decreasing penalty strength per iteration for efficient regularization (Blumenthal et al., 6 Aug 2025, Rosenzweig et al., 2017).

Bilevel and Multiscale Approaches

Bilevel regularized inversion is essential when the regularized problem is itself an optimal parameter identification under PDE constraints, as in hidden-law discovery in reaction–diffusion systems. Sequential initialization of the lower-level PDE solve accelerates convergence and enables multiscale effects, with Landweber iterations efficiently propagating regularization throughout the hierarchy (Nguyen, 2024).

4. Selection and Adaptation of Regularization Parameters

Regularization parameter choice is critical for stability. Techniques include:

• Morozov's discrepancy principle: $\lambda$ is selected so the data misfit matches the known noise variance (Hannah et al., 2012, Swanson et al., 2017).
• L-curve and generalized cross-validation: trade-off curves in ($\|Ax-b\|, \|Lx\|$)-space to locate a stable corner as optimal $\lambda$ (Anikin et al., 2024).
• Unbiased Predictive Risk Estimator (UPRE): grid-search or optimization over $\alpha$ (regularization parameter) in the IRLS scheme for focused inversion (Vatankhah et al., 2017).
• Adaptive, data-driven strategies, e.g., plug-and-play adjust denoising strength, while in bilevel approaches stopping and penalty decays are coupled to residuals and scale sequences (Luiken et al., 2024, Nguyen, 2024).

5. Specialized Algorithms and Applications

Regularized inversion is central across domains:

• Plasma physics: Poisson-regularized inversion for Bremsstrahlung spectra recovers EEDFs with high fidelity in experimental and synthetic regimes (Swanson et al., 2017).
• Remote sensing and astronomical imaging: Regularized inversion with accurate instrument models and spatial smoothness restores super-resolution beyond standard coaddition, balancing offset correction, noise suppression, and physical fidelity (Orieux et al., 2011).
• MRI: Regularized Nonlinear Inversion (NLINV and SMS‐NLINV) jointly estimates coil sensitivities and image content in parallel imaging without explicit calibration, integrating phase–pole corrections to guarantee smooth, artifact-free sensitivity maps (Blumenthal et al., 6 Aug 2025, Rosenzweig et al., 2017).
• Compressed sensing: GAN-based generative priors with regularized training of intermediate layers enable intermediate-layer inversion schemes (such as ILO and mGANprior-RTIL) with lower representation error and substantially improved recovery of natural images (Gunn et al., 2022).
• Kohn-Sham DFT: Moreau–Yosida (proximal point) regularized inversion converts the mapping from density to Kohn-Sham potential into a smooth, convex problem with provable convergence and error bounds (Herbst et al., 2024).

6. Computational Complexity and Efficiency Considerations

Computational efficiency is ensured by:

• Exploiting randomization (RSVD) in large-scale linear problems to restrict computation to dominant spectral subspaces, with error controlled by the sketch dimension and explicit error estimates under source conditions (Ito et al., 2019, Vatankhah et al., 2017).
• Using iterative solvers (CG, LSQR) and recursive update formulas that reduce per-step memory and computational requirements to the cost of matrix-vector products and short-term storage (Vatankhah et al., 2017, Chung et al., 2016).
• Plug-and-play frameworks enabling the use of pretrained denoisers or deep networks as regularization modules leverage highly optimized network inference routines within classical optimization steps (Luiken et al., 2024).

7. Theoretical Guarantees and Error Analysis

Regularized inversion algorithms admit rigorous theoretical analysis in many settings:

• Convergence and stability can be established for Landweber and superiorized variants under bounded perturbations and early stopping (Gibali et al., 2023).
• Error control for randomized algorithms is achieved via source conditions, with explicit expressions for error propagation under truncation, approximation, and noise (Ito et al., 2019).
• In bilevel and multiscale algorithms, tangential cone conditions and a posteriori error criteria guarantee regularization in the presence of inexact or sequentially updated PDE solvers (Nguyen, 2024).
• In Kohn-Sham inversion via Moreau–Yosida regularization, Lipschitz properties and contraction results yield explicit, verifiable error bounds sensitive to density perturbations (Herbst et al., 2024).

---

The regularized inversion algorithm framework unifies classical and modern methods for solving ill-posed inverse problems with explicit guarantees, efficient implementation, and flexibility for advanced domains and priors. Advances in algorithmic design, parameter selection, and application-specific adaptation continue to expand the power and theoretical reach of regularized inversion across scientific and engineering fields.