---
title: Tikhonov Guidance for Ill-posed Inverse Problems
url: https://www.emergentmind.com/topics/tikhonov-guidance
type: topic
---

# Tikhonov Guidance for Ill-posed Inverse Problems

Tikhonov guidance denotes the body of parameter-choice rules, stopping criteria, and iterative constructions that make Tikhonov regularization operational for ill-posed inverse problems with noisy data. In the linear Hilbert-space setting, it concerns equations \(Ax=y\) with only \(y^\delta\) available, and it includes single-shot Tikhonov, iterated and nonstationary iterated schemes, discrepancy-based rules, heuristic rules for unknown noise, and Krylov-projected implementations. A particularly explicit recent formulation is the range-relaxed nonstationary iterated Tikhonov method, which chooses iteration-dependent Lagrange multipliers so that the next residual lies in a feasible interval rather than at one prescribed value [2011.05372].

## 1. Problem class and basic regularization model

The standard setting uses separable Hilbert spaces \(X\) and \(Y\), a bounded linear operator \(A:X\to Y\), and noisy data \(y^\delta\in Y\) satisfying
\[
\|y-y^\delta\|_Y \le \delta,
\]
with known noise level \(\delta>0\). Exact solvability is assumed in the sense that there exists \(x^\star\in X\) with \(Ax^\star=y\), while ill-posedness means that \(A^{-1}:\operatorname{Range}(A)\to X\) is either not defined or discontinuous. The distinguished exact solution is the minimal-norm solution \(x^\dagger=A^\dagger y\), and an optional source condition of the form \(x^\dagger\in \operatorname{Range}((A^*A)^\nu)\), \(\nu>0\), is used to discuss rates [2011.05372].

The one-shot Tikhonov problem minimizes
\[
J_\alpha(x)=\|Ax-y^\delta\|_Y^2+\alpha\|x\|_X^2,
\]
with normal equation
\[
(A^*A+\alpha I)x_\alpha=A^*y^\delta.
\]
This formulation stabilizes the inversion by balancing fidelity and norm control. In the same setting, residual level sets
\[
\Omega_\mu:=\{x\in X:\|Ax-y^\delta\|_Y\le \mu\},\qquad \mu\ge 0,
\]
are closed and convex, satisfy \(\Omega_\mu\subset \Omega_{\mu'}\) for \(\mu\le \mu'\), contain \(A^{-1}(y)\) whenever \(\mu\ge \delta\), and have nonempty interior if \(\mu>\delta\) [2011.05372].

This geometric viewpoint is central. It turns parameter choice into the problem of determining how aggressively one should move an iterate toward a residual level set that is compatible with the noise floor. That interpretation is what underlies range-relaxed Tikhonov guidance.

## 2. Nonstationary iterated Tikhonov and the range-relaxed rule

Iterated Tikhonov replaces the one-shot penalty \(\|x\|_X^2\) by a proximal penalty around the current iterate:
\[
x_{k+1}\in \arg\min_{x\in X}\|Ax-y^\delta\|_Y^2+\alpha_k\|x-x_k\|_X^2.
\]
Its optimality condition is
\[
(A^*A+\alpha_k I)x_{k+1}=A^*y^\delta+\alpha_k x_k.
\]
If \(\alpha_k\equiv \alpha\), one obtains stationary iterated Tikhonov; if \(\alpha_k\) varies, one obtains nonstationary iterated Tikhonov. Writing \(\lambda_k:=1/\alpha_k\), the update becomes
\[
x_{k+1}
= x_k-\lambda_k (I+\lambda_k A^*A)^{-1}A^*(Ax_k-y^\delta),
\]
so \(\lambda_k\) acts as a Lagrange multiplier in a constrained projection step [2011.05372].

A common a priori choice is geometric growth, \(\lambda_k=q^k\), equivalently \(\alpha_k=\alpha_0 q^{-k}\). The cited analysis identifies the usual trade-off: this can be fast, but it may be unstable if \(q\) is too large or too slow if \(q\) is too small. Exact a posteriori target-residual prescriptions avoid that rigidity, but they require solving a nonlinear scalar equation at every iteration [2011.05372].

The range-relaxed rule addresses both issues by requiring only that the next residual remain in an admissible interval:
\[
\delta \le \|Ax_{k+1}-y^\delta\|_Y \le p\|Ax_k-y^\delta\|_Y+(1-p)\delta,
\qquad p\in(0,1).
\]
Thus the new iterate is not forced to hit one exact residual target. Instead, it is accepted whenever its residual lies between the noise floor and a relaxed contraction of the current residual. Geometrically, \(x_{k+1}\) is an orthogonal projection of \(x_k\) onto some \(\Omega_{\mu_k}\) with
\[
\delta \le \mu_k \le p\|Ax_k-y^\delta\|_Y+(1-p)\delta,
\qquad
x_{k+1}=P_{\Omega_{\mu_k}}(x_k).
\]
The corresponding constrained problem is
\[
\min_x \|x-x_k\|_X^2
\quad\text{subject to}\quad
\|Ax-y^\delta\|_Y\le \mu,
\]
and its Lagrangian, for fixed \(\mu\), is
\[
\mathcal{L}(x,\lambda)
=
\frac{\lambda}{2}\big(\|Ax-y^\delta\|_Y^2-\mu^2\big)
+
\frac12\|x-x_k\|_X^2.
\]
For \(\lambda>0\), the minimizer is
\[
x'(\lambda)=x_k-\lambda (I+\lambda A^*A)^{-1}A^*(Ax_k-y^\delta),
\]
with \(\mu=\|Ax'(\lambda)-y^\delta\|_Y\) [2011.05372].

Defining
\[
\pi_k(\lambda):=x_k-\lambda (I+\lambda A^*A)^{-1}A^*(Ax_k-y^\delta),
\qquad
G_k(\lambda):=\|A\pi_k(\lambda)-y^\delta\|_Y^2,
\]
the practical rule becomes
\[
\delta^2\le G_k(\lambda_k)\le \big[p\|Ax_k-y^\delta\|_Y+(1-p)\delta\big]^2.
\]
A decisive structural fact is that \(G_k\) is strictly decreasing on \((0,\infty)\), because
\[
\frac{d}{d\lambda}G_k(\lambda)
=
-2\left\langle
A^*(A\pi_k(\lambda)-y^\delta),
(I+\lambda A^*A)^{-1}A^*(A\pi_k(\lambda)-y^\delta)
\right\rangle
\le 0.
\]
Hence the admissible \(\lambda_k\) form an interval \([\lambda_{\min},\lambda_{\max}]\) with \(0<\lambda_{\min}<\lambda_{\max}\le\infty\). This interval-valued admissibility is the defining feature of range-relaxed guidance [2011.05372].

## 3. Convergence, stability, and discrepancy stopping

The range-relaxed criterion implies a direct residual contraction law:
\[
\|Ax_{k+1}-y^\delta\|_Y-\delta
\le
p\big(\|Ax_k-y^\delta\|_Y-\delta\big),
\]
and therefore
\[
\|Ax_k-y^\delta\|_Y-\delta
\le
p^k\big(\|Ax_0-y^\delta\|_Y-\delta\big).
\]
This gives exponential decay of the “distance to the noise floor” and immediately implies that Morozov discrepancy stopping,
\[
k^*=\min\{k\ge 0:\|Ax_k-y^\delta\|_Y\le \tau\delta\},\qquad \tau>1,
\]
is finite. The stopping index obeys the explicit bound
\[
k^*
\le
|\ln p|^{-1}
\ln
\left[
\frac{\|Ax_0-y^\delta\|_Y-\delta}{(\tau-1)\delta}
\right]
+1
\]
[2011.05372].

The same analysis yields a monotonic error identity. Writing \(r(x):=\|Ax-y^\delta\|_Y^2\), one has
\[
\|x^\star-x_k\|_X^2-\|x^\star-x_{k+1}\|_X^2
=
\|x_{k+1}-x_k\|_X^2
+\lambda_k\|A(x^\star-x_{k+1})\|_Y^2
+\lambda_k\big[r(x_{k+1})-r(x^\star)\big].
\]
Thus the squared error decreases strictly. Before discrepancy is met, the gain is bounded below by
\[
\|x^\star-x_k\|_X^2-\|x^\star-x_{k+1}\|_X^2
\ge
\lambda_k^2\|A^*(Ax_{k+1}-y^\delta)\|_X^2
+\lambda_k\big(\|A(x^\star-x_{k+1})\|_Y^2+(\tau^2-1)\delta^2\big),
\]
which identifies both primal movement and residual curvature as sources of descent [2011.05372].

In the exact-data case \(\delta=0\), the lower bound
\[
\lambda_k \ge \frac{1-p}{\|A\|^2}
\]
implies \(\sum_k \lambda_k=\infty\). By the Brill–Schock criterion for nonstationary iterated Tikhonov, this guarantees strong convergence of the iterates to a solution of \(Ax=y\) [2011.05372].

Under the source condition \(x^\dagger\in \operatorname{Range}((A^*A)^\nu)\), the cited work aligns the range-relaxed analysis with classical Tikhonov balance arguments. Single-shot Tikhonov obeys
\[
\|x_\alpha-x^\dagger\|_X
=
O(\alpha^\nu)+O(\delta/\sqrt{\alpha}),
\]
and the range-relaxed nonstationary method is compatible with convergence \(x_{k(\delta)}\to x^\dagger\) as \(\delta\to 0\), with order-optimal scaling recovered when the effective parameter at stopping satisfies the usual balance \(\alpha\sim \delta^{2/(2\nu+1)}\) [2011.05372].

## 4. Algorithmic realization and parameter heuristics

The implementable form of range-relaxed nonstationary iterated Tikhonov takes as input \(A\), \(A^*\), \(y^\delta\), the noise level \(\delta\), an initial guess \(x_0\), a relaxation parameter \(p\in(0,1)\), and a discrepancy factor \(\tau>1\). Typical values reported for the relaxation are \(p\in[0.1,0.2]\), with \(p\in[0.1,0.3]\) described as a good trade-off between fast residual decrease and robust feasibility; typical discrepancy factors are \(\tau\in[2,3]\) in practical guidance, with \(\tau\in[1.5,3]\) also discussed in relation to computational cost and accuracy [2011.05372].

At iteration \(k\), one first checks discrepancy. If \(\|Ax_k-y^\delta\|_Y\le \tau\delta\), the process stops. Otherwise, one defines the upper residual target
\[
\theta_k:=p\|Ax_k-y^\delta\|_Y+(1-p)\delta
\]
and searches for \(\lambda_k>0\) satisfying
\[
\delta^2\le G_k(\lambda_k)\le \theta_k^2.
\]
A practical initialization is the lower bound
\[
\lambda_{k,0}
=
\frac{(\|Ax_k-y^\delta\|_Y-\theta_k)\|Ax_k-y^\delta\|_Y}
{\|A^*(Ax_k-y^\delta)\|_X^2}.
\]
For \(k\ge 2\), two heuristics are explicitly suggested: reusing \(\lambda_{k-1}\), or extrapolating on the logarithmic scale,
\[
\lambda_{k,0}:=\frac{\lambda_{k-1}^2}{\lambda_{k-2}}.
\]
The inner solve is a monotone Newton-like iteration on \(G_k\):
\[
\lambda_{k,j+1}
=
\lambda_{k,j}
-\omega_j\frac{G_k(\lambda_{k,j})}{G_k'(\lambda_{k,j})},
\]
with \(\omega_0=1\), doubling \(\omega_j\) when \(G_k(\lambda_{k,j-1})>2\theta_k^2\), and otherwise setting \(\omega_j=1\). The inner loop stops as soon as
\[
\delta^2\le G_k(\lambda_{k,j})\le \theta_k^2.
\]
The reported behavior is that the inner Newton loop typically needs only a few steps, often \(1\)–\(3\), because the residual map is monotone and exact target matching is unnecessary [2011.05372].

Each evaluation of \(\pi_k(\lambda)\) and \(G_k(\lambda)\) requires solving
\[
(I+\lambda A^*A)u=\text{rhs},
\]
equivalently \((A^*A+\alpha I)u=\text{rhs}\) with \(\alpha=1/\lambda\). The cited implementation guidance recommends preconditioned CG, multigrid, FFT-based solvers for convolution operators, or direct factorizations, depending on the application class [2011.05372].

The method’s practical advantages and limitations are sharply delimited. The feasible interval \([\lambda_{\min},\lambda_{\max}]\) protects against overshooting the residual target, unlike pure geometric growth \(\lambda_k=q^k\), which may become unstable for large \(q\). At the same time, the method does not remove the cost of solving \((A^*A+\alpha I)\) systems, and if those systems dominate runtime or are poorly conditioned, preconditioning remains essential. Very large noise or very small \(\tau\) can narrow the feasible interval and increase inner iterations, while poor initial guesses can increase \(k^*\) [2011.05372].

## 5. Empirical behavior in deblurring and inverse potential identification

The reported numerical study evaluates range-relaxed nonstationary iterated Tikhonov against geometric nonstationary iterated Tikhonov and an a posteriori Donatelli–Hanke variant in two distinct settings: image deblurring and a two-dimensional inverse potential problem. The metrics are relative reconstruction error \(\|x^\star-x_k\|/\|x^\star\|\), residual \(\|Ax_k-y^\delta\|\), and the number of linear systems solved as a proxy for cost. The central empirical findings are exponential residual decay consistent with theory, fewer total linear solves than the a posteriori Donatelli–Hanke method, and robustness in low-noise regimes [2011.05372].

| Application and noise | Setup | Total linear systems |
|---|---|---|
| Deblurring, \(\delta_{\mathrm{rel}}=10^{-1}\%\) | \(p=0.2\), \(\tau=3\), \(x_0=y^\delta\) | gNIT 6; Donatelli–Hanke 15; rrNIT 7 |
| Deblurring, \(\delta_{\mathrm{rel}}=10^{-3}\%\) | same | gNIT 17; Donatelli–Hanke 23; rrNIT 11 |
| Deblurring, \(\delta_{\mathrm{rel}}=10^{-6}\%\) | same | gNIT 36; Donatelli–Hanke 43; rrNIT 16 |
| Inverse potential, \(\delta_{\mathrm{rel}}=10^{-1}\%\) | \(p=0.1\), \(\tau=3\), \(x_0\equiv 1.5\) | gNIT 6; Donatelli–Hanke 11; rrNIT 6 |
| Inverse potential, \(\delta_{\mathrm{rel}}=10^{-3}\%\) | same | gNIT 10; Donatelli–Hanke 34; rrNIT 10 |
| Inverse potential, \(\delta_{\mathrm{rel}}=10^{-6}\%\) | same | gNIT 13; Donatelli–Hanke 86; rrNIT 12 |

In the deblurring experiment, \(A\) is a space-invariant convolution with Gaussian point-spread function on a \(256\times 256\) image, implemented with FFT-based \(A\) and \(A^*\); the Gaussian point-spread function uses size \(257\times 257\) and \(\sigma=4\). In the inverse potential problem, the forward PDE is
\[
-\Delta u = x \ \text{in}\ \Omega=(0,1)^2,\qquad
u=0 \ \text{on}\ \partial\Omega,
\]
with
\[
A:L_2(\Omega)\to L_2(\partial\Omega),\qquad
Ax=\partial_\nu u|_{\partial\Omega}.
\]
The unknown source has sharp gradients, so the problem probes whether the method remains effective beyond smooth-image benchmarks [2011.05372].

Two empirical observations are especially relevant for parameter guidance. First, \(\lambda_k\) tends to grow exponentially across iterations, so log-extrapolation for \(\lambda_{k,0}\) substantially reduces inner Newton effort. Second, the gain over exact target-residual a posteriori rules becomes more pronounced at very low noise, because accepting any \(\lambda_k\) inside a feasible interval is cheaper than solving a scalar nonlinear equation to high precision at every step [2011.05372].

## 6. Position within the broader Tikhonov parameter-choice literature

Range-relaxed nonstationary iterated Tikhonov is one member of a larger family of Tikhonov guidance strategies. In Krylov-subspace settings, one influential approach embeds the discrepancy principle inside Arnoldi–Tikhonov by estimating the unknown noise norm through the GMRES residual plateau,
\[
\widehat{\delta}_m=\phi_{m-1}(0)=\|r_{m-1}\|_2,
\]
and choosing \(\lambda_m\) in reduced space so that
\[
\phi_m(\lambda_m)=\tau \widehat{\delta}_m,
\]
thereby turning parameter selection into a low-dimensional update that reuses Arnoldi quantities [1307.0334]. A complementary Morozov-based line of work formulates the Tikhonov solution and the regularization parameter as a single nonlinear system and solves it with a projected Newton direction computed in a Krylov subspace, coupled with an Armijo line search; this yields a globally convergent noise-constrained algorithm for large-scale problems [1910.08432].

When the data noise level is unknown but an operator perturbation bound \(\eta\) is available, semi-heuristic rules modify standard heuristic functionals such as heuristic discrepancy, Hanke–Raus, or quasi-optimality by subtracting compensators like \(D\eta\|x_{\alpha,\delta,\eta}\|\) or \(D\eta/\sqrt{\alpha}\), and by enforcing a lower search bound \(\gamma(\eta)\); these rules are designed precisely for the case in which operator error does not satisfy the same irregularity assumptions as data noise [1807.05042]. Under white Gaussian noise, a different route is to minimize a lower bound of the predictive risk,
\[
T_{(\rho^2,\sigma^2)}(\alpha),
\]
whose minimizer depends only on the signal-to-noise ratio; when \(\sigma^2\) and \(\rho^2\) are unknown, the iterative I-PRO scheme alternates between regularization-parameter minimization and signal-to-noise-ratio estimation [1912.13004].

Heuristic rules remain important when no reliable noise bound is available, but they are not interchangeable. For quasi-optimality, one detailed analysis shows that the global minimizer can fail badly, whereas one proper local minimizer of the quasi-optimality function is always a good regularization parameter in the finite-dimensional standard Tikhonov setting [1708.02149]. In models with several competing structural priors, Tikhonov guidance becomes vector-valued: multi-parameter Tikhonov regularization uses several penalties simultaneously and selects the parameter vector by discrepancy or balancing principles, with the balancing system
\[
\gamma \eta_1 \psi_1(x^\delta_\eta)=\gamma \eta_2 \psi_2(x^\delta_\eta)=\phi(x^\delta_\eta,y^\delta)
\]
as the central equilibrium condition [1102.1173].

The notion also extends beyond linear Hilbert-space inverse problems. For nonlinear problems in Banach spaces, Bregman-distance penalization yields stability, convergence, and rates in Bregman distances, and supports an iterated Tikhonov method with discrepancy stopping [2012.11017]. In Banach-space oversmoothing, where the exact solution lies outside the penalty space, convergence can still be proved in weaker norms, with rates of the form
\[
\|x^\delta_{\kappa_{DP}}-x^\dagger\|_X \le c\,\delta^{\theta/(a+\theta)}
\]
under sectorial-operator and interpolation-scale assumptions [2008.07681]. In Krylov hybridization, CG–Tikhonov can be interpreted as a filtration of the CG Lanczos vectors, with Lanczos filters \(\gamma_i^{(m)}(c)\) damping later, noise-amplifying components more strongly as \(c\to\infty\) [2505.24056], and the iterated Golub–Kahan–Tikhonov method achieves
\[
O\!\left(\delta^{2i/(2i+1)}\right)
\]
under the source conditions and balancing strategy stated in its analysis [2507.12307].

Taken together, these developments indicate that Tikhonov guidance is not a single rule but a structured design problem. The relevant choice depends on what is known about the noise, whether the forward operator is perturbed, whether the problem is reduced in a Krylov subspace, whether multiple penalties are used, and whether the underlying geometry is Hilbertian, Banach, or projection-based. Within that landscape, the range-relaxed criterion stands out for replacing exact target matching by a guaranteed feasible interval, thereby linking geometric projection, robust convergence, and low inner-solve complexity in a particularly direct way [2011.05372].

Source: https://www.emergentmind.com/topics/tikhonov-guidance