---
title: Slowly Annealed Langevin Dynamics (SALD)
url: https://www.emergentmind.com/topics/slowly-annealed-langevin-dynamics-sald
type: topic
---

# Slowly Annealed Langevin Dynamics (SALD)

Slowly Annealed Langevin Dynamics (SALD) denotes a class of Langevin-based procedures that move through a sequence of intermediate target distributions, typically by performing several stochastic refinement steps at each level of a decreasing annealing schedule before passing the terminal state forward to the next, harder level. In the literature, the annealed quantity is not unique: it may be the Gaussian smoothing level of a data distribution, a posterior measurement-noise variance, a temperature, or a smoothing covariance in a multimodal approximation path. Several relevant papers therefore use the term *annealed Langevin dynamics* rather than *SALD* explicitly, while still implementing the multi-level, warm-started, gradual-refinement pattern usually associated with SALD [2010.02977] [2510.26324] [2601.22349].

## 1. Concept and defining pattern

In its standard score-based form, SALD operates on a ladder of noise levels \(\sigma_1 > \cdots > \sigma_L\), with repeated Langevin steps at each level and a transition from broad, high-noise distributions to sharper, low-noise ones. VoiceGrad gives a particularly clear task-adapted instance: it learns a score approximator for Gaussian-corrupted target-speaker mel-spectrogram distributions and, at test time, updates an input mel-spectrogram by annealed Langevin dynamics, moving it “towards the nearest stationary point of the target distribution” [2010.02977]. In its DSM formulation, the conversion update is
\[
\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z},
\qquad
\gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2},
\qquad
\mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),
\]
run for \(T\) iterations at each level \(l\) [2010.02977].

The rationale for annealing in this setting is the standard score-based modeling argument that real data lie near a low-dimensional manifold in a high-dimensional ambient space. High-noise levels produce smoother densities that “fill the whole space” more effectively, whereas low-noise levels approach the true data distribution. This gives SALD its characteristic exploration-to-refinement behavior: early stages promote mobility across broad regions, while late stages sharpen the sample near the target manifold [2010.02977].

A second, more formal continuous-time interpretation appears in diffusion annealed Langevin theory. There, one prescribes an interpolation
\[
\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,
\]
between a target law \(\pi\) and a base law \(\nu\), and then considers the score-driven diffusion
\[
dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,
\]
where \(\hat p_t\) is the time-rescaled family of intermediate marginals. In that formulation, “slowly annealed” is quantified by a parameter \(\kappa\in(0,1)\), since \(\hat p_t=p_{\kappa t}\); small \(\kappa\) means that the target marginal family evolves slowly relative to diffusion time [2511.10406].

## 2. Annealed quantities and posterior variants

A central feature of SALD is that the annealed variable need not be the same from one domain to another. In standard score-based generation, the annealed variable is the Gaussian smoothing level. In posterior sampling for linear inverse problems, however, the annealed variable can instead be the observation-noise variance. The posterior-sampling algorithm of “Posterior Sampling by Combining Diffusion Models with Annealed Langevin Dynamics” constructs a decreasing measurement-noise schedule
\[
\eta_1 > \eta_2 > \cdots > \eta_N = \eta
\]
and defines coupled auxiliary measurements
\[
y_i = y_{i+1} + \mathcal N\!\left(0,(\eta_i^2-\eta_{i+1}^2)I_m\right),
\]
so that each \(y_i\) induces an intermediate posterior \(p(x\mid y_i)\). The method then runs short Langevin transitions from \(p(x\mid y_i)\) toward \(p(x\mid y_{i+1})\), using the posterior score decomposition
\[
\nabla_x \log p(x\mid y) = s(x) + \frac{A^\top(y-Ax)}{\eta^2}.
\]
The paper is explicit that this is “effectively a form of slowly annealed Langevin dynamics,” but not standard ALD in the usual diffusion-model sense, because the annealing is over posterior likelihood noise rather than prior smoothing scale [2510.26324].

Linear inverse problems also motivate annealing over perturbation noise injected directly into the unknown variable. “Solving Linear Inverse Problems using Higher-Order Annealed Langevin Diffusion” introduces levels
\[
\{\sigma_l\}_{l=1}^{L+1},\qquad \sigma_1 > \sigma_2 > \cdots > \sigma_L > \sigma_{L+1}=0,
\]
with perturbed variables
\[
\tilde{x}_l = x + e_l,\qquad e_l\sim \mathcal{N}(0,\sigma_l^2 I).
\]
At each level, the target is the annealed posterior over \(\tilde{x}_l\), and the sampler can be overdamped, underdamped, or third-order, with preconditioning and warm starts across levels. The same annealed continuation also makes discrete unknowns tractable, because the discrete prior becomes a continuous Gaussian mixture and its score is available through Tweedie’s identity
\[
\nabla_{\tilde{x}_l}\log p(\tilde{x}_l) = \frac{\mathbb{E}_{\sigma_l}[x\mid \tilde{x}_l]-\tilde{x}_l}{\sigma_l^2}.
\]
This places SALD squarely inside Bayesian inverse problems, not only unconditional generative modeling [2305.05014].

Massive MIMO detection provides a parallel discrete-posterior example. There the unknown symbol vector \(x\in \mathcal X^{N_u}\) has a finite-constellation prior, so the prior score is undefined on the original discrete support. The detector therefore introduces
\[
\tilde{x}_l = x+\nu_l,\qquad \nu_l\sim\mathcal N(0,\sigma_l^2 I),
\]
derives a continuous prior score from a Gaussian-mixture denoiser, and uses a time-inhomogeneous unadjusted Langevin algorithm in the spectral domain. The resulting chain explores a sequence of smooth surrogate posteriors before a final projection back to the constellation. The paper explicitly presents this as an annealed Langevin method for a discrete posterior, and the later underdamped extension embeds momentum into the same multi-level structure [2205.05776] [2210.15071].

## 3. Slow annealing as a convergence mechanism

Several papers make the role of slow annealing mathematically explicit. In the kinetic-theory treatment of simulated annealing, the annealed parameter is the temperature \(T(t)\), and the canonical slow-cooling schedule is logarithmic:
\[
T(t)=\frac{T_0}{\log(2+t)}.
\]
The entropy computation for the corresponding linear kinetic equation produces an extra term involving \(T'(t)/T(t)^2\), and the paper states that one needs
\[
T'(t)=o(T^2(t))
\]
as \(T(t)\to 0\). This is why \(T(t)\approx 1/\log t\) is singled out as “slow enough,” whereas \(T(t)\approx 1/t\) is not [2401.05553].

A closely related mean-field result appears in “Mean-Field Langevin Dynamics: Exponential Convergence and Annealing.” For the McKean–Vlasov dynamics
\[
dX_t = - \nabla V[\mu_t](X_t)\,dt + \sqrt{2\tau}\, dB_t,
\qquad
\mu_t=\mathrm{Law}(X_t),
\]
the fixed-temperature free energy \(F_\tau(\mu)=G(\mu)+\tau H(\mu)\) converges exponentially fast under a uniform family of log-Sobolev inequalities. More importantly for SALD, the annealed dynamics with
\[
\tau_t = \frac{\alpha}{\log t}
\qquad \text{for some } \alpha>\alpha^*
\]
satisfies
\[
G(\mu_t)-\inf G \le C' \frac{\log\log t}{\log t}.
\]
The result is a mean-field analogue of classical simulated annealing: slow logarithmic cooling yields convergence in objective value to the global minimum of the unregularized functional [2202.01009].

Time-inhomogeneous Langevin theory provides an abstract non-asymptotic formulation of the same idea. For
\[
d X_t = -\nabla U_{\tau(t)}(X_t)\,dt + \sqrt{2}\, d W_t,
\]
the forward-KL derivative to the moving target \(\pi_{\tau(t)}\) is
\[
\frac{d}{d t}\KL(\mu_t|\pi_{\tau(t)}) =
-\int q_t\biggl|\nabla \log\frac{q_t}{p_{\tau(t)}}\biggr|^2
-\int q_t\partial_t\log p_{\tau(t)}\,d\lambda.
\]
This yields a non-asymptotic decomposition of the final error into initialization error, a tracking term involving \(|\dot\tau(t)|\), discretization error for Euler–Maruyama, and a terminal mismatch term \(c\tau(t)\). In that framework, “slow annealing” means that the target moves slowly enough for the diffusion to keep up with the instantaneous equilibrium [2601.22349].

Diffusion annealed Langevin theory refines this picture by comparing the practical explicit-score diffusion to a Nelson process that realizes the prescribed marginal path exactly. If \(\hat p_t=p_{\kappa t}\), then the path-space KL satisfies
\[
d_{KL}(Q_X,Q_Y)= \frac{\kappa}{4} \, \int_0^T \, |p'_t|^2 \, dt,
\]
and consequently
\[
d_{KL}(\tilde \pi,\mathcal L(Y_{T/\kappa})) \leq \frac{\kappa}{4} \, \int_0^T  \, |p'_t|^2 \, dt.
\]
Here the small parameter \(\kappa\) is a direct measure of slowness, and stronger logarithmic Sobolev control improves this basic \(O(\kappa)\)-type bias estimate [2511.10406].

## 4. High-dimensional SALD, preconditioning, and discretization

Recent theory emphasizes that in high-dimensional and infinite-dimensional regimes, slow annealing alone is not the whole story. “Dimension-Free Multimodal Sampling via Preconditioned Annealed Langevin Dynamics” studies continuous-time ALD for Gaussian-mixture targets
\[
\rho_\star^d = \sum_{i\in I} w_i\,\mathcal N(m_i^d,\Sigma_i^d)
\]
along the de-smoothing path
\[
\rho_t^d = \rho_\star^d * \mathcal N\!\Big(0,\frac{T-t}{T}C^d\Big),
\qquad
C^d=\operatorname{Diag}(\lambda_1,\dots,\lambda_d),
\]
with preconditioned dynamics
\[
dX_t^d = \Gamma^d \nabla\log\rho_t^d(X_t^d)\,dt +\sqrt{2\Gamma^d}\,dW_t^d,
\qquad
\Gamma^d=\operatorname{Diag}(\gamma_1,\dots,\gamma_d).
\]
The main spectral quantity is
\[
\mathcal K_d := \frac{1}{16}\sum_{i\in I} w_i \sum_{j=1}^d \frac{\lambda_j}{\gamma_j} \log\!\Big(1+\frac{\lambda_j}{\sigma_{ij}}\Big),
\]
and choosing
\[
T^d = \epsilon^{-1}\mathcal K_d
\]
guarantees
\[
\mathrm{KL}\!\big(\rho_\star^d\,\|\,\rho_{T^d}^{\mathrm{ALD},d}\big)\le \epsilon.
\]
Under misspecified scores, the paper derives explicit weighted summability conditions showing that a sufficiently decaying preconditioner spectrum is needed to prevent error accumulation across coordinates and preserve dimension-uniform control [2602.01449].

The companion discretization analysis shows that numerical integration can fundamentally alter which slow-annealing regimes are actually usable. For Gaussian mixtures, the annealed score decomposes as
\[
\nabla\log\rho_t^d(x) = -(B_t^d)^{-1}x+G_t^d(x),
\]
where the diagonal linear part is stiff in high-frequency coordinates. If Euler–Maruyama treats that part explicitly, stability requires
\[
h\,\sup_{n,j}\frac{\gamma_j}{\underline{\sigma}_j+\kappa_{t_n}\lambda_j}\le 2.
\]
Combined with the continuous-time annealing-bias condition, this can force the initial smoothed law to remain uniformly close to the target across dimensions, which is contrary to the usual SALD motivation of starting from a much smoother distribution. The paper therefore develops an exact-linear-part exponential-integrator scheme and proves the dimension-uniform bound
\[
\sup_{d\ge1} \operatorname{KL}\!\bigl(\rho_\star^d\,\|\,\operatorname{Law}(Y_T^d)\bigr)
\le
\frac{1}{8T}\sum_{j\ge1}\frac{\lambda_j^2}{\gamma_j\underline{\sigma}_j}
+
C_{\mathrm{disc}}(1+T^2)h_{\max}.
\]
This shows that sufficiently slow annealing can be made dimension-uniformly accurate, but only with a discretization that handles the stiff linear structure appropriately [2605.16473].

## 5. Applications and empirical behavior

VoiceGrad illustrates SALD as a conditional refinement mechanism rather than unconditional generation. The state variable is the normalized \(80\)-dimensional log mel-spectrogram, initialized from source speech rather than Gaussian noise, and the DSM version uses \(L=21\) noise levels with \(\sigma_1=1.2\), \(\sigma_L=0.01\), \(L'=4\), \(\varepsilon=10^{-5}\), and \(T=32\), yielding \(576\) annealed Langevin updates. The paper reports that DSM-based VoiceGrad required \(576\) iterations whereas the DPM version required \(11\), that “starting the iteration from a certain point in the middle rather than from the beginning is effective in terms of the audio quality of the converted speech,” and that BNF conditioning strongly improves performance [2010.02977].

Posterior SALD also appears as a warm-started refinement stage for image inverse problems. On FFHQ-256, the posterior-sampling paper evaluates inpainting, \(4\times\) super-resolution, and Gaussian deblurring on \(1\text{k}\) validation images, using initial reconstructions \(x_0\) from DPS and then refining near \(p(x\mid x_0,y)\) with its annealed Langevin sampler. The reported metrics are per-image \(L^2\) distance to ground truth and FID. Increasing annealed Langevin time decreases \(L^2\) but often increases FID; in inpainting, sufficiently small step sizes let the method beat DPS on both metrics, and qualitative samples preserve ground-truth attributes better. The experiments use the local or warm-start variant rather than posterior sampling from scratch [2510.26324].

In communications and inverse problems, SALD-like methods are tightly connected to discrete structure. The annealed Langevin MIMO detector uses \(L=20\) levels, \(\sigma_1=1\), \(\sigma_{20}=0.01\), and \(T=70\), for a total of \(1400\) Langevin steps, and the paper reports that at least \(L=10\) levels are needed to match state-of-the-art detectors. The underdamped extension shows that low-complexity settings benefit substantially from momentum: the paper states that total iterations \(L\times T\) can be reduced by about a factor of \(10\) relative to the previous overdamped setup, from \(1400\) to \(150\), while still yielding superior low-budget SER [2205.05776] [2210.15071].

Higher-order annealed Langevin diffusion extends the same continuation idea to second-order and third-order samplers. In MIMO detection, with enough levels the overdamped, underdamped, and third-order variants perform similarly, but with fewer levels “underdamped and especially third-order outperform overdamped.” In channel estimation, third-order with only \(L=58\) gets close to an overdamped baseline using \(L=2311\) for \(\mathrm{SNR}>0\) dB, and among low-complexity methods the higher-order dynamics are best. On image tasks, the reported metrics are PSNR, LPIPS, and FID, and third-order Langevin often outperforms SNIPS and other baselines in noiseless or high-SNR settings, especially on deblurring and inpainting [2305.05014].

SALD has also been adapted from sampling to black-box optimization. In AI-aided programmable-channel optimization, the objective is converted into a Gibbs distribution
\[
P(\phi\mid u)\propto e^{\alpha \cdot \mathcal{F}(\mathcal{M}_u(\phi))},
\]
and the annealed Langevin update becomes
\[
\phi_k \leftarrow \phi_{k-1}
+ \frac{\epsilon}{2\sigma_t^2}\Big(D_\theta(\phi_{k-1};u,\sigma_t)-\phi_{k-1}\Big)
+ \sqrt{\epsilon}\, z.
\]
The deployed system uses \(T=10\) annealing time steps and \(K=5\) iterations per step. In the reported latency table for optimizing \(8\) configurations, Algorithm 1 ALD achieves latency \(0.145\) s and rate \(0.414\), compared with \(473.57\) s and \(0.369\) for ZOGD, \(125.19\) s and \(0.453\) for simulator perfect knowledge, and \(119.56\) s and \(0.372\) for simulator imperfect knowledge [2510.18978].

## 6. Scope, misconceptions, and limitations

A persistent misconception is that SALD names a single canonical algorithm. The surveyed literature suggests a broader picture. In standard score-based generation, annealing is over Gaussian smoothing levels; in posterior refinement it may be over measurement-noise variance; in simulated annealing and mean-field annealing it is temperature; in Gaussian-mixture analyses it is a smoothing covariance path. This suggests that SALD is better understood as a common continuation pattern—successive Langevin corrections through nearby targets—rather than as one fixed mathematical object [2510.26324] [2202.01009].

A second misconception is that any score-based annealing procedure is equivalent to reverse-time diffusion sampling. That is explicitly false in compositional SBI: the paper on compositional scores argues that reverse-SDE sampling is structurally biased when the aggregated score does not equal the score of the true forward-noised multi-observation posterior, whereas annealed Langevin dynamics is principled because it treats the composite score as the exact score of a different, tractable bridge family [2605.21253].

Theoretical guarantees are also narrower than empirical usage sometimes suggests. The posterior-sampling polynomial-time guarantee under an \(L^4\) score-error bound is proved for \(\alpha\)-strongly log-concave priors with \(L\)-Lipschitz score, with a local-log-concavity extension after localization; it is not a general non-log-concave posterior-sampling theorem [2510.26324]. The forward-KL analysis of time-inhomogeneous Langevin diffusions assumes differentiable, dissipative, globally Lipschitz drifts along the annealing path [2601.22349]. The dimension-uniform multimodal analyses are proved for Gaussian mixtures with diagonal or co-diagonalizable covariance structure [2602.01449] [2605.16473]. The diffusion annealed Langevin study is continuous-time and exact-score, not a theory of practical learned-score discretizations [2511.10406].

Finally, slow annealing is not sufficient by itself. High-dimensional discretization theory shows that a naive Euler–Maruyama implementation can destroy the benefit of starting from a strongly smoothed initial law, whereas a stiff-aware exponential integrator preserves the intended annealed behavior [2605.16473]. In that sense, the modern SALD literature frames annealing speed, score quality, preconditioning, and numerical integration as coupled design variables rather than separable ingredients.

Source: https://www.emergentmind.com/topics/slowly-annealed-langevin-dynamics-sald