---
title: Preconditioned Annealed Langevin Dynamics
url: https://www.emergentmind.com/topics/preconditioned-annealed-langevin-dynamics-ald
type: topic
---

# Preconditioned Annealed Langevin Dynamics

Preconditioned Annealed Langevin Dynamics (ALD) denotes a class of samplers in which Langevin updates are executed along an annealed sequence of intermediate target distributions while both the drift and the injected Gaussian noise are scaled by a positive-definite matrix or operator. Across recent work, the annealing variable may be a diffusion prior scale, an effective measurement-noise level, or an explicit Gaussian smoothing covariance; the preconditioner may be constant, scale-dependent, time-dependent, position-dependent, or defined directly in function space. The common objective is to stabilize sampling on ill-conditioned or multimodal posteriors without discarding the posterior structure encoded by the likelihood and prior, particularly in inverse problems, high-dimensional Gaussian-mixture approximations, and communication systems [2512.05791][2605.06091].

## 1. Core formulation

In the linear-Gaussian inverse-problem setting, the posterior has the standard form
\[
p(x\mid y)\propto p(y\mid x)\,p(x),
\]
with posterior score
\[
\nabla_x \log p(x\mid y)=\nabla_x \log p(x)+A^\top \Sigma^{-1}(y-Ax)
\]
in the real-valued case, or the analogous Hermitian form in complex MRI models [2510.26324][2512.05791]. Preconditioned ALD replaces the isotropic Langevin step by a metric-scaled update. In one widely used formulation,
\[
x_{k+1}^{(i)}=x_k^{(i)}+\eta_i M_i\!\left[\nabla_x \log p(y\mid x_k^{(i)})+s_{\sigma_i}(x_k^{(i)})\right]+\sqrt{2\eta_i}\,M_i^{1/2}\xi_k,
\]
where $s_{\sigma_i}(x)=\nabla_x\log q_{\sigma_i}(x)$ is the score of a diffused prior and $M_i\succ 0$ is a scale-dependent preconditioner; setting $M_i=I$ recovers standard ALD or ULA [2512.05791].

The annealed targets themselves vary across the literature. In diffusion posterior sampling for MRI, the target at scale $\sigma$ is
\[
p_\sigma(x\mid y)\propto p(y\mid x)\,q_\sigma(x),
\]
so the likelihood remains exact while the prior is Gaussian-smoothed across reverse-diffusion scales [2512.05791]. In the log-concave conditional-sampling theory of diffusion-plus-Langevin methods, annealing is instead imposed on the measurement variance: one constructs a sequence of auxiliary measurements $y_k\sim Ax+\mathcal N(0,\eta_k^2I)$ and runs short Langevin chains that move from $p(x\mid y_k)$ to $p(x\mid y_{k+1})$ [2510.26324]. In continuous-time multimodal analyses, annealing appears as an explicit smoothing path
\[
\rho_t^d=\rho_\star^d * \mathcal N\!\left(0,\frac{T-t}{T}C^d\right),
\]
with preconditioned diffusion run against the score of the smoothed law [2602.01449].

A recurring distinction is between annealing the likelihood and preconditioning the dynamics. Exact-likelihood variants keep $p(y\mid x)$ unchanged at all scales and rely on preconditioning to absorb the resulting stiffness, whereas classical annealed-likelihood methods soften the data term at high noise levels to reduce the effective Lipschitz constant [2512.05791]. This separation is central to recent exact-posterior formulations.

## 2. Preconditioner design and geometric interpretation

The most common preconditioner in inverse problems approximates the inverse curvature of the negative log-posterior. In MRI diffusion posterior sampling, the proposed choice is
\[
M_\sigma=(A^HA+\sigma^{-2}I)^{-1},
\]
motivated by the heuristic Hessian approximation
\[
-\nabla^2 \log p_\sigma(x\mid y)\approx A^HA+\sigma^{-2}I.
\]
In that construction, directions strongly constrained by the data receive smaller steps, while weakly constrained directions receive larger steps proportional to $\sigma^2$; the paper explicitly interprets this as Newton-like or natural-gradient preconditioning [2512.05791].

Spectral preconditioning is especially explicit in massive MIMO detection. There, ALD is run in the singular-vector coordinates of the channel matrix, and the update uses a diagonal, singular-value-aware preconditioner $\Lambda_l$ that scales both the deterministic score step and the injected Gaussian noise. The resulting mass matrix is constant within each annealing level, preserves the tempered smoothed posterior at that level, and is designed to compensate for channel anisotropy in the spectral domain [2205.05776].

In infinite-dimensional linear Bayesian inverse problems, preconditioning is derived mode by mode. For Gaussian priors and diagonalized forward operators, the optimal observed-mode eigenvalues are
\[
\lambda_j^{(0)}=\big[\mu_j^{-1}+\sigma^{-2}(A_N^\top A_N)_{jj}\big]^{-1}, \qquad j\le N,
\]
while on unobserved modes one obtains $\lambda_j^{(0)}=\mu_j$; first-order corrections depend explicitly on the score approximation error coefficients. The paper’s stated criterion is uniform convergence across posterior modes, with the optimal preconditioner depending on both the forward operator and score error [2505.18276].

A more general geometry arises when the preconditioner depends on time and position. TIPreL considers the SDE
\[
dX_t=\big[-B(t,X_t)\nabla \Psi(X_t)+\operatorname{div} B(t,X_t)\big]dt+\sqrt{2}\,B^{1/2}(t,X_t)\,dW_t,
\]
with $B(t,x)$ symmetric positive definite. The divergence correction is required whenever the diffusion depends on position in order to preserve $\pi(x)\propto e^{-\Psi(x)}$ as invariant density; if $B$ is independent of $x$, the correction vanishes [2605.06091]. This is the main formal distinction between constant-metric preconditioned ALD and genuinely Riemannian or manifold-type variants.

## 3. Continuous-time theory, convergence, and discretization

Theoretical analyses of ALD focus on two intertwined questions: whether the annealed process remains close to the intended path of intermediate distributions, and whether its discretization remains stable under refinement. In the log-concave posterior-sampling theory for diffusion-assisted inverse problems, the main positive result is that annealed Langevin dynamics with short stagewise mixing yields polynomial-time conditional sampling under only an $L^4$ score-error bound, rather than the sub-exponential MGF control needed by classical long-run Langevin analyses. Under global strong log-concavity, the stagewise mixing time is
\[
T_k=O\!\left(\frac{m\gamma_k+\log(\lambda/\varepsilon)}{\alpha}\right),
\qquad
\gamma_k=\left(\frac{\eta_k}{\eta_{k+1}}\right)^2-1,
\]
for transitions between consecutive measurement-noise levels [2510.26324].

For multimodal targets approximated by Gaussian mixtures, preconditioning acquires an explicitly spectral role. Continuous-time dimension-free control is obtained when
\[
\sup_d \mathcal K_d<\infty,
\qquad
\mathcal K_d=\frac{1}{16}\sum_{i\in I} w_i \sum_{j=1}^d \frac{\lambda_j}{\gamma_j}\log\!\left(1+\frac{\lambda_j}{\sigma_{ij}}\right),
\]
with a sufficient condition
\[
\sum_{i\in I} w_i \sum_{j\ge 1}\frac{\lambda_j^2}{\gamma_j \sigma_{ij}}<\infty.
\]
Here the smoothing spectrum $\lambda_j$, the preconditioner spectrum $\gamma_j$, and the mixture covariances $\sigma_{ij}$ must be jointly balanced so that the required annealing time horizon does not deteriorate with dimension [2602.01449].

Discretization turns out to be highly scheme-dependent. For Euler–Maruyama applied to preconditioned ALD on Gaussian mixtures, the explicit stability constraint is
\[
h\,\sup_{n,j}\frac{\gamma_j}{\underline{\sigma}_j+\kappa_{t_n}\lambda_j}\le 2.
\]
The same paper shows that this condition can force the initial smoothed law to remain uniformly close to the target across dimensions. By contrast, an exponential-integrator or exact-linear-part scheme integrates the stiff linear component exactly and admits the dimension-uniform bound
\[
\sup_{d\ge 1}\mathrm{KL}\big(\pi^d \,\|\, \mathrm{Law}(Y_T^d)\big)
\le
\frac{1}{8T}\sum_{j\ge 1}\frac{\lambda_j^2}{\gamma_j \underline{\sigma}_j}
+
C_{\mathrm{disc}}(1+T^2)h_{\max},
\]
which can be made arbitrarily small uniformly in dimension by increasing the annealing time and refining the mesh [2605.16473]. A plausible implication is that several apparent “ALD limitations” are actually discretization artifacts rather than intrinsic failures of annealing.

Time-inhomogeneous preconditioned Langevin theory supplies a complementary perspective. TIPreL proves exponential convergence in continuous time in Kullback–Leibler divergence under a preconditioned log-Sobolev inequality and establishes $W_2$ convergence for a tamed Euler discretization with time- and space-dependent diffusion coefficients [2605.06091]. Diffusion-annealed Langevin theory, framed via Nelson processes, yields a path-space KL bias bound of the form
\[
d_{\mathrm{KL}}(\tilde \pi,\mathrm{Law}(Y_{T/\kappa}))
\le
\frac{\kappa}{4}\int_0^T |p_t'|^2\,dt,
\]
and shows that replacing a Poincaré inequality by a logarithmic Sobolev inequality improves the efficiency of the model [2511.10406].

## 4. MRI and linear inverse problems

MRI reconstruction provides one of the clearest practical realizations of preconditioned ALD. The MRI posterior sampler of “Fast and Robust Diffusion Posterior Sampling for MR Image Reconstruction Using the Preconditioned Unadjusted Langevin Algorithm” keeps the exact complex Gaussian likelihood at every reverse-diffusion noise scale and uses
\[
M_i=(A^HA+\sigma_i^{-2}I)^{-1}
\]
to neutralize the ill-conditioning induced by the MRI forward operator. The paper reports that, for posterior sampling in Cartesian and non-Cartesian accelerated MRI, the new approach outperforms annealed sampling in reconstruction speed and sample quality; in Cartesian brain MRI it yields higher PSNR/SSIM and cleaner error maps than annealed likelihood, despite fewer iterations per scale ($K=4$ versus $K=8$), and it remains stable across coil-compression settings and radial acquisitions. A single fixed step size $\gamma=0.5$ is used across all experiments, with no tuning required [2512.05791].

The same theme reappears in function-space Bayesian inverse problems. In the Hilbert-space analysis of score-based priors, preconditioned Langevin dynamics is formulated directly on the infinite-dimensional state space, and the paper proves the existence and form of an optimal preconditioner that guarantees a uniform convergence rate across all posterior modes. Preventing numerical instabilities requires preconditioning, because the preconditioner must be trace-class to keep the $C$-Wiener process and the corresponding Langevin dynamics well defined as the discretization is refined [2505.18276].

Higher-order formulations extend the preconditioned ALD idea beyond overdamped dynamics. For linear inverse problems, second-order and third-order preconditioned Langevin diffusions are constructed with exact invariant Gibbs distributions matching the non-preconditioned targets, and practical implementations are obtained through splitting schemes such as ABO, BAOAB, and BACOCAB. The paper reports that these higher-order, preconditioned, annealed samplers achieve strong performance on MIMO symbol detection, channel estimation, and image inverse problems while maintaining comparable or lower computational complexity than competing approaches [2305.05014].

## 5. Communications, channel geometry, and programmable environments

In communications, preconditioned ALD has been used most explicitly for massive MIMO detection. The detector of “Annealed Langevin Dynamics for Massive MIMO Detection” defines a smoothed posterior over discrete constellations, runs the dynamics in the channel’s SVD coordinates, and employs a diagonal spectral preconditioner that rescales both drift and diffusion. The paper states that this yields state-of-the-art symbol error rate performance, while the unfolded robust variant becomes noise-variance agnostic by replacing the closed-form likelihood score with a learned module [2205.05776].

Annealing also appears in wireless-channel optimization framed as posterior-like sampling over programmable parameters. In “AI-Aided Annealed Langevin Dynamics for Rapid Optimization of Programmable Channels,” the reported inference algorithm is unpreconditioned, but the paper explicitly formulates the preconditioned extension
\[
\theta_{k+1}=\theta_k+\frac{\epsilon_k}{2}M_k\nabla_\theta \log p_{\sigma_t}(\theta_k\mid x)+\sqrt{\epsilon_k}\,M_k^{1/2}\xi_k
\]
and proposes diagonal adaptive, Fisher-information, Hessian-based, and per-RIS block choices for $M_k$. Those designs are presented as natural extensions that could improve conditioning and reduce tuning sensitivity, but they are not tested in the reported experiments [2510.18978].

These communication-oriented examples are notable because they broaden the role of ALD beyond posterior sampling from image priors. One strand uses preconditioning to cope with anisotropic likelihood geometry induced by a known forward operator; another treats ALD as a stochastic optimizer over a surrogate posterior and leaves preconditioning as an explicit open design variable. This suggests a widening interpretation of preconditioned ALD as a geometry-aware stochastic flow rather than a method tied to a single application domain.

## 6. Limitations, misconceptions, and open directions

A common misconception is that “annealing” has a single meaning within ALD. The cited literature uses at least three distinct annealing mechanisms: diffusion-prior scales with exact likelihood, decreasing effective measurement noise, and explicit Gaussian smoothing paths of the target density [2512.05791][2510.26324][2602.01449]. Consequently, comparisons between ALD methods are only meaningful after specifying which intermediate densities are being targeted.

A second misconception is that any metric scaling can be inserted into Langevin dynamics without changing the target distribution. That is true for constant symmetric positive-definite preconditioners when the noise covariance is matched to the metric, but it is false in the position-dependent case unless the divergence correction is included. TIPreL makes this point explicit: omitting $\operatorname{div} B(t,x)$ from a state-dependent diffusion generally destroys exact invariance [2605.06091].

The practical cost of preconditioning is also nontrivial. In MRI, each pULA iteration includes approximately $10$ conjugate-gradient iterations to apply the preconditioner, although the overall runtime still decreases because fewer network evaluations and fewer iterations are required [2512.05791]. In time- and space-dependent schemes, computing $\operatorname{div} B(t,x)$ and scalable inverse-Hessian surrogates is identified as a major bottleneck, and efficient approximations to the divergence are singled out as a priority for future work [2605.06091]. In dimension-uniform Gaussian-mixture analyses, the discretization scheme itself determines whether stiffness is manageable; exponential integrators succeed in regimes where Euler–Maruyama imposes prohibitive constraints [2605.16473].

Open directions stated across the papers include learned preconditioners and learned schedules in deep-unfolded ALD, manifold-aware SDEs for constrained variables such as phases, end-to-end theory for moving-target annealing with time- and space-dependent metrics, and scalable approximations for divergence and curvature in high dimension [2510.18978][2605.06091]. Another persistent limitation is model mismatch: exact-likelihood samplers still assume an accurate forward operator and a sufficiently accurate score model, and severe mismatch can degrade reconstruction or sampling quality [2512.05791][2505.18276].

Taken together, these developments position preconditioned ALD as a technically heterogeneous but conceptually coherent framework: annealing defines a sequence of tractable intermediate laws, while preconditioning reshapes the Langevin geometry so that those laws can be explored rapidly, stably, and, in favorable settings, with guarantees that persist under ill-conditioning, multimodality, and refinement of the ambient dimension.

Source: https://www.emergentmind.com/topics/preconditioned-annealed-langevin-dynamics-ald