Slowly Annealed Langevin Dynamics (SALD)

• Slowly Annealed Langevin Dynamics (SALD) is a technique that refines samples by sequential Langevin updates over a ladder of decreasing noise or temperature levels.
• It employs a gradual annealing schedule to transition from broad exploration at high noise levels to precise refinement near data manifolds.
• SALD is applied in diverse fields such as voice conversion, image restoration, and MIMO detection, with advancements in preconditioning and discretization ensuring robust convergence.

Slowly Annealed Langevin Dynamics (SALD) denotes a class of Langevin-based procedures that move through a sequence of intermediate target distributions, typically by performing several stochastic refinement steps at each level of a decreasing annealing schedule before passing the terminal state forward to the next, harder level. In the literature, the annealed quantity is not unique: it may be the Gaussian smoothing level of a data distribution, a posterior measurement-noise variance, a temperature, or a smoothing covariance in a multimodal approximation path. Several relevant papers therefore use the term annealed Langevin dynamics rather than SALD explicitly, while still implementing the multi-level, warm-started, gradual-refinement pattern usually associated with SALD (Kameoka et al., 2020, Xun et al., 30 Oct 2025, Habring et al., 29 Jan 2026).

1. Concept and defining pattern

In its standard score-based form, SALD operates on a ladder of noise levels $\sigma_1 > \cdots > \sigma_L$, with repeated Langevin steps at each level and a transition from broad, high-noise distributions to sharper, low-noise ones. VoiceGrad gives a particularly clear task-adapted instance: it learns a score approximator for Gaussian-corrupted target-speaker mel-spectrogram distributions and, at test time, updates an input mel-spectrogram by annealed Langevin dynamics, moving it “towards the nearest stationary point of the target distribution” (Kameoka et al., 2020). In its DSM formulation, the conversion update is

$$\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z}, \qquad \gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2}, \qquad \mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),$$

run for $T$ iterations at each level $l$ (Kameoka et al., 2020).

The rationale for annealing in this setting is the standard score-based modeling argument that real data lie near a low-dimensional manifold in a high-dimensional ambient space. High-noise levels produce smoother densities that “fill the whole space” more effectively, whereas low-noise levels approach the true data distribution. This gives SALD its characteristic exploration-to-refinement behavior: early stages promote mobility across broad regions, while late stages sharpen the sample near the target manifold (Kameoka et al., 2020).

A second, more formal continuous-time interpretation appears in diffusion annealed Langevin theory. There, one prescribes an interpolation

$$\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,$$

between a target law $\pi$ and a base law $\nu$, and then considers the score-driven diffusion

$$dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,$$

where $\hat p_t$ is the time-rescaled family of intermediate marginals. In that formulation, “slowly annealed” is quantified by a parameter $\kappa\in(0,1)$, since $$\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z}, \qquad \gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2}, \qquad \mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),$$0; small $$\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z}, \qquad \gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2}, \qquad \mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),$$1 means that the target marginal family evolves slowly relative to diffusion time (Cattiaux et al., 13 Nov 2025).

2. Annealed quantities and posterior variants

A central feature of SALD is that the annealed variable need not be the same from one domain to another. In standard score-based generation, the annealed variable is the Gaussian smoothing level. In posterior sampling for linear inverse problems, however, the annealed variable can instead be the observation-noise variance. The posterior-sampling algorithm of “Posterior Sampling by Combining Diffusion Models with Annealed Langevin Dynamics” constructs a decreasing measurement-noise schedule

$$\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z}, \qquad \gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2}, \qquad \mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),$$2

and defines coupled auxiliary measurements

$$\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z}, \qquad \gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2}, \qquad \mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),$$3

so that each $$\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z}, \qquad \gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2}, \qquad \mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),$$4 induces an intermediate posterior $$\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z}, \qquad \gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2}, \qquad \mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),$$5. The method then runs short Langevin transitions from $$\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z}, \qquad \gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2}, \qquad \mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),$$6 toward $$\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z}, \qquad \gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2}, \qquad \mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),$$7, using the posterior score decomposition

$$\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z}, \qquad \gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2}, \qquad \mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),$$8

The paper is explicit that this is “effectively a form of slowly annealed Langevin dynamics,” but not standard ALD in the usual diffusion-model sense, because the annealing is over posterior likelihood noise rather than prior smoothing scale (Xun et al., 30 Oct 2025).

Linear inverse problems also motivate annealing over perturbation noise injected directly into the unknown variable. “Solving Linear Inverse Problems using Higher-Order Annealed Langevin Diffusion” introduces levels

$$\mathbf{x} \leftarrow \mathbf{x} - \frac{\gamma_l}{\sigma_l}\boldsymbol{\epsilon}_{\theta}(\mathbf{x}, l, k) +\sqrt{2\gamma_l} \mathbf{z}, \qquad \gamma_l = \varepsilon \cdot \frac{\sigma_l^2}{\sigma_L^2}, \qquad \mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),$$9

with perturbed variables

$T$0

At each level, the target is the annealed posterior over $T$1, and the sampler can be overdamped, underdamped, or third-order, with preconditioning and warm starts across levels. The same annealed continuation also makes discrete unknowns tractable, because the discrete prior becomes a continuous Gaussian mixture and its score is available through Tweedie’s identity

$T$2

This places SALD squarely inside Bayesian inverse problems, not only unconditional generative modeling (Zilberstein et al., 2023).

Massive MIMO detection provides a parallel discrete-posterior example. There the unknown symbol vector $T$3 has a finite-constellation prior, so the prior score is undefined on the original discrete support. The detector therefore introduces

$T$4

derives a continuous prior score from a Gaussian-mixture denoiser, and uses a time-inhomogeneous unadjusted Langevin algorithm in the spectral domain. The resulting chain explores a sequence of smooth surrogate posteriors before a final projection back to the constellation. The paper explicitly presents this as an annealed Langevin method for a discrete posterior, and the later underdamped extension embeds momentum into the same multi-level structure (Zilberstein et al., 2022, Zilberstein et al., 2022).

3. Slow annealing as a convergence mechanism

Several papers make the role of slow annealing mathematically explicit. In the kinetic-theory treatment of simulated annealing, the annealed parameter is the temperature $T$5, and the canonical slow-cooling schedule is logarithmic: $T$6 The entropy computation for the corresponding linear kinetic equation produces an extra term involving $T$7, and the paper states that one needs

$T$8

as $T$9. This is why $l$0 is singled out as “slow enough,” whereas $l$1 is not (Pareschi, 2024).

A closely related mean-field result appears in “Mean-Field Langevin Dynamics: Exponential Convergence and Annealing.” For the McKean–Vlasov dynamics

$l$2

the fixed-temperature free energy $l$3 converges exponentially fast under a uniform family of log-Sobolev inequalities. More importantly for SALD, the annealed dynamics with

$l$4

satisfies

$l$5

The result is a mean-field analogue of classical simulated annealing: slow logarithmic cooling yields convergence in objective value to the global minimum of the unregularized functional (Chizat, 2022).

Time-inhomogeneous Langevin theory provides an abstract non-asymptotic formulation of the same idea. For

$l$6

the forward-KL derivative to the moving target $l$7 is

$l$8

This yields a non-asymptotic decomposition of the final error into initialization error, a tracking term involving $l$9, discretization error for Euler–Maruyama, and a terminal mismatch term $$\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,$$0. In that framework, “slow annealing” means that the target moves slowly enough for the diffusion to keep up with the instantaneous equilibrium (Habring et al., 29 Jan 2026).

Diffusion annealed Langevin theory refines this picture by comparing the practical explicit-score diffusion to a Nelson process that realizes the prescribed marginal path exactly. If $$\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,$$1, then the path-space KL satisfies

$$\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,$$2

and consequently

$$\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,$$3

Here the small parameter $$\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,$$4 is a direct measure of slowness, and stronger logarithmic Sobolev control improves this basic $$\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,$$5-type bias estimate (Cattiaux et al., 13 Nov 2025).

4. High-dimensional SALD, preconditioning, and discretization

Recent theory emphasizes that in high-dimensional and infinite-dimensional regimes, slow annealing alone is not the whole story. “Dimension-Free Multimodal Sampling via Preconditioned Annealed Langevin Dynamics” studies continuous-time ALD for Gaussian-mixture targets

$$\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,$$6

along the de-smoothing path

$$\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,$$7

with preconditioned dynamics

$$\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,$$8

The main spectral quantity is

$$\tilde X_t= \sqrt{\lambda_t} \, X + \sqrt{1-\lambda_t} \, Z,$$9

and choosing

$\pi$0

guarantees

$\pi$1

Under misspecified scores, the paper derives explicit weighted summability conditions showing that a sufficiently decaying preconditioner spectrum is needed to prevent error accumulation across coordinates and preserve dimension-uniform control (Baldassari et al., 1 Feb 2026).

The companion discretization analysis shows that numerical integration can fundamentally alter which slow-annealing regimes are actually usable. For Gaussian mixtures, the annealed score decomposes as

$\pi$2

where the diagonal linear part is stiff in high-frequency coordinates. If Euler–Maruyama treats that part explicitly, stability requires

$\pi$3

Combined with the continuous-time annealing-bias condition, this can force the initial smoothed law to remain uniformly close to the target across dimensions, which is contrary to the usual SALD motivation of starting from a much smoother distribution. The paper therefore develops an exact-linear-part exponential-integrator scheme and proves the dimension-uniform bound

$\pi$4

This shows that sufficiently slow annealing can be made dimension-uniformly accurate, but only with a discretization that handles the stiff linear structure appropriately (Baldassari et al., 15 May 2026).

5. Applications and empirical behavior

VoiceGrad illustrates SALD as a conditional refinement mechanism rather than unconditional generation. The state variable is the normalized $\pi$5-dimensional log mel-spectrogram, initialized from source speech rather than Gaussian noise, and the DSM version uses $\pi$6 noise levels with $\pi$7, $\pi$8, $\pi$9, $\nu$0, and $\nu$1, yielding $\nu$2 annealed Langevin updates. The paper reports that DSM-based VoiceGrad required $\nu$3 iterations whereas the DPM version required $\nu$4, that “starting the iteration from a certain point in the middle rather than from the beginning is effective in terms of the audio quality of the converted speech,” and that BNF conditioning strongly improves performance (Kameoka et al., 2020).

Posterior SALD also appears as a warm-started refinement stage for image inverse problems. On FFHQ-256, the posterior-sampling paper evaluates inpainting, $\nu$5 super-resolution, and Gaussian deblurring on $\nu$6 validation images, using initial reconstructions $\nu$7 from DPS and then refining near $\nu$8 with its annealed Langevin sampler. The reported metrics are per-image $\nu$9 distance to ground truth and FID. Increasing annealed Langevin time decreases $$dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,$$0 but often increases FID; in inpainting, sufficiently small step sizes let the method beat DPS on both metrics, and qualitative samples preserve ground-truth attributes better. The experiments use the local or warm-start variant rather than posterior sampling from scratch (Xun et al., 30 Oct 2025).

In communications and inverse problems, SALD-like methods are tightly connected to discrete structure. The annealed Langevin MIMO detector uses $$dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,$$1 levels, $$dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,$$2, $$dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,$$3, and $$dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,$$4, for a total of $$dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,$$5 Langevin steps, and the paper reports that at least $$dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,$$6 levels are needed to match state-of-the-art detectors. The underdamped extension shows that low-complexity settings benefit substantially from momentum: the paper states that total iterations $$dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,$$7 can be reduced by about a factor of $$dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,$$8 relative to the previous overdamped setup, from $$dY_t = \sqrt 2 \, dB_t + \nabla \ln \hat p_{t}(Y_t) \, dt,$$9 to $\hat p_t$0, while still yielding superior low-budget SER (Zilberstein et al., 2022, Zilberstein et al., 2022).

Higher-order annealed Langevin diffusion extends the same continuation idea to second-order and third-order samplers. In MIMO detection, with enough levels the overdamped, underdamped, and third-order variants perform similarly, but with fewer levels “underdamped and especially third-order outperform overdamped.” In channel estimation, third-order with only $\hat p_t$1 gets close to an overdamped baseline using $\hat p_t$2 for $\hat p_t$3 dB, and among low-complexity methods the higher-order dynamics are best. On image tasks, the reported metrics are PSNR, LPIPS, and FID, and third-order Langevin often outperforms SNIPS and other baselines in noiseless or high-SNR settings, especially on deblurring and inpainting (Zilberstein et al., 2023).

SALD has also been adapted from sampling to black-box optimization. In AI-aided programmable-channel optimization, the objective is converted into a Gibbs distribution

$\hat p_t$4

and the annealed Langevin update becomes

$\hat p_t$5

The deployed system uses $\hat p_t$6 annealing time steps and $\hat p_t$7 iterations per step. In the reported latency table for optimizing $\hat p_t$8 configurations, Algorithm 1 ALD achieves latency $\hat p_t$9 s and rate $\kappa\in(0,1)$0, compared with $\kappa\in(0,1)$1 s and $\kappa\in(0,1)$2 for ZOGD, $\kappa\in(0,1)$3 s and $\kappa\in(0,1)$4 for simulator perfect knowledge, and $\kappa\in(0,1)$5 s and $\kappa\in(0,1)$6 for simulator imperfect knowledge (Shaked et al., 21 Oct 2025).

6. Scope, misconceptions, and limitations

A persistent misconception is that SALD names a single canonical algorithm. The surveyed literature suggests a broader picture. In standard score-based generation, annealing is over Gaussian smoothing levels; in posterior refinement it may be over measurement-noise variance; in simulated annealing and mean-field annealing it is temperature; in Gaussian-mixture analyses it is a smoothing covariance path. This suggests that SALD is better understood as a common continuation pattern—successive Langevin corrections through nearby targets—rather than as one fixed mathematical object (Xun et al., 30 Oct 2025, Chizat, 2022).

A second misconception is that any score-based annealing procedure is equivalent to reverse-time diffusion sampling. That is explicitly false in compositional SBI: the paper on compositional scores argues that reverse-SDE sampling is structurally biased when the aggregated score does not equal the score of the true forward-noised multi-observation posterior, whereas annealed Langevin dynamics is principled because it treats the composite score as the exact score of a different, tractable bridge family (Touron et al., 20 May 2026).

Theoretical guarantees are also narrower than empirical usage sometimes suggests. The posterior-sampling polynomial-time guarantee under an $\kappa\in(0,1)$7 score-error bound is proved for $\kappa\in(0,1)$8-strongly log-concave priors with $\kappa\in(0,1)$9-Lipschitz score, with a local-log-concavity extension after localization; it is not a general non-log-concave posterior-sampling theorem (Xun et al., 30 Oct 2025). The forward-KL analysis of time-inhomogeneous Langevin diffusions assumes differentiable, dissipative, globally Lipschitz drifts along the annealing path (Habring et al., 29 Jan 2026). The dimension-uniform multimodal analyses are proved for Gaussian mixtures with diagonal or co-diagonalizable covariance structure (Baldassari et al., 1 Feb 2026, Baldassari et al., 15 May 2026). The diffusion annealed Langevin study is continuous-time and exact-score, not a theory of practical learned-score discretizations (Cattiaux et al., 13 Nov 2025).

Finally, slow annealing is not sufficient by itself. High-dimensional discretization theory shows that a naive Euler–Maruyama implementation can destroy the benefit of starting from a strongly smoothed initial law, whereas a stiff-aware exponential integrator preserves the intended annealed behavior (Baldassari et al., 15 May 2026). In that sense, the modern SALD literature frames annealing speed, score quality, preconditioning, and numerical integration as coupled design variables rather than separable ingredients.