---
title: 'VA-SALD: Velocity-Aware Guided Diffusion'
url: https://www.emergentmind.com/topics/velocity-aware-sald-va-sald
type: topic
---

# VA-SALD: Velocity-Aware Guided Diffusion

Velocity-Aware SALD (VA-SALD) is a guided sampling framework for pretrained diffusion-based and related generative models that extends Slowly Annealed Langevin Dynamics (SALD) by explicitly incorporating the transport velocity of the pretrained marginal path and using time slowdown to control guidance-induced deviation. In the formulation developed in "Slowly Annealed Langevin Dynamics: Theory and Applications to Training-Free Guided Generation" [2605.07950], VA-SALD is designed for training-free guided generation: it uses a fixed pretrained score model, a guide or reward \(f_t\), and a slowdown schedule \(t=t(s)\) to approximate a terminal guided target \(\pi_T\), while providing non-asymptotic convergence guarantees in KL divergence.

## 1. SALD, moving targets, and path velocity

SALD begins from a nonstationary sampling problem. One is given a path of target distributions \((\pi_t)_{t\in[0,T]}\) on \(\mathbb{R}^d\), typically with oracle access to the time-dependent score \(\nabla \log \pi_t(x)\), and seeks to approximately sample from \(\pi_T\). Standard annealed Langevin dynamics along the path is
\[
dX_t=\nabla\log \pi_t(X_t)\,dt+\sqrt{2}\,dW_t.
\]
SALD slows this traversal by introducing algorithmic time \(s\in[0,S]\) and a monotone schedule \(t=t(s)\), with slowed path \(\tilde\pi_s=\pi_{t(s)}\). The continuous-time SALD dynamics are
\[
dX_s=\nabla\log \tilde\pi_s(X_s)\,ds+\sqrt{2}\,dW_s,\qquad X_0\sim \rho_0,
\]
and the Euler–Maruyama discretization is
\[
X_{k+1}^\eta=X_k^\eta+\eta\,\nabla\log \pi_{t_k}(X_k^\eta)+\sqrt{2\eta}\,\xi_k,\qquad \xi_k\sim \mathcal N(0,I_d).
\]
A standard choice is linear slowdown \(t(s)=s/r\), so the same physical path is traversed in \(r\) times more algorithmic time [2605.07950].

The paper’s notion of “velocity” is not a particle kinematic observable but a transport velocity field for a curve of measures. For a path \((\mu_t)\), a vector field \(u_t\) is a transport velocity if
\[
\partial_t\mu_t=-\nabla\cdot(\mu_t u_t).
\]
This continuity-equation viewpoint quantifies how fast probability mass moves along the path. The paper defines the exponential-moment functional
\[
\mathfrak E_\alpha(\mu_t,v_t)=\frac{1}{\alpha}\log \mathbb E_{\mu_t}\!\left[\exp(\alpha\|v_t(X)\|^2)\right]
\]
and the corresponding path complexity
\[
\mathcal A_\alpha(\mu,v)=\int_0^T \mathfrak E_\alpha(\mu_t,v_t)\,dt.
\]
SALD’s convergence depends on this complexity and on functional inequalities, especially log-Sobolev and Poincaré inequalities, along the intermediate targets. A common misconception is that slowdown is merely a heuristic increase in step count; in the paper’s formulation, it is a continuous-time reparametrization whose benefit is expressed analytically through contraction and path-complexity terms.

## 2. Construction of VA-SALD for guided diffusion models

VA-SALD specializes SALD to the setting of pretrained Itô diffusions and guided generation. The forward diffusion is
\[
dY_\tau=\bar B_\tau(Y_\tau)\,d\tau+\bar \sigma_\tau\,dW_\tau,\qquad \tau\in[0,T],\quad Y_0\sim q_0=p_{\mathrm{data}},
\]
with reverse marginal family
\[
p_t=q_{T-t},\qquad B_t=\bar B_{T-t},\qquad \sigma_t=\bar \sigma_{T-t}.
\]
The associated forward and reverse transport velocities are
\[
\bar u_\tau(y)=\bar B_\tau(y)-\frac{\bar\sigma_\tau^2}{2}\nabla\log q_\tau(y),\qquad
u_t(x)=-\bar u_{T-t}(x)=-B_t(x)+\frac{\sigma_t^2}{2}\nabla\log p_t(x).
\]
Guidance is introduced through a potential or reward \(f\), or more generally a time-dependent guide \(f_t\) with \(f_T=f\), defining
\[
\pi_t(x)\propto p_t(x)\exp(-f_t(x)).
\]
The framework assumes oracle access to \(\nabla \log p_t(x)\) and to \(\nabla f_t(x)\), or to a stochastic zeroth-order estimate in black-box guidance settings [2605.07950].

VA-SALD combines slowdown with explicit use of the pretrained velocity \(u_t\). With schedule \(t=t(s)\), \(\dot t(s)>0\), and \(\tilde\pi_s=\pi_{t(s)}\), the SDE is
\[
\begin{aligned}
dX_s
&=\left(\dot t(s)\,u_{t(s)}(X_s)+\frac{\sigma_{t(s)}^2}{2}\nabla\log p_{t(s)}(X_s)-\frac{\sigma_{t(s)}^2}{2}\nabla f_{t(s)}(X_s)\right)ds+\sigma_{t(s)}\,dW_s \\
&=\left(\dot t(s)\,u_{t(s)}(X_s)+\frac{\sigma_{t(s)}^2}{2}\nabla\log \pi_{t(s)}(X_s)\right)ds+\sigma_{t(s)}\,dW_s.
\end{aligned}
\]
Equivalently, using the explicit form of \(u_t\),
\[
dX_s=\left(-\dot t(s)B_{t(s)}(X_s)+(\dot t(s)+1)\frac{\sigma_{t(s)}^2}{2}\nabla\log p_{t(s)}(X_s)-\frac{\sigma_{t(s)}^2}{2}\nabla f_{t(s)}(X_s)\right)ds+\sigma_{t(s)}\,dW_s.
\]

The defining distinction from SALD is structural. SALD uses only \(\nabla\log \pi_{t(s)}\), whereas VA-SALD uses both the pretrained path velocity \(u_t\) and a Langevin drift based on \(\nabla\log \pi_t\). In the unguided case \(f_t\equiv 0\), so \(\pi_t\equiv p_t\), the paper shows that if \(X_0\sim p_0\), then \(\mathrm{Law}(X_s)=p_{t(s)}\) exactly for any schedule \(t(s)\). Slowdown therefore does not itself introduce bias in the unguided setting; VA-SALD exactly tracks the pretrained marginal path [2605.07950]. This property is the basis of the “velocity-aware” terminology: the base transport is handled explicitly, and only the guidance-induced deviation must be corrected.

## 3. Guided-path deviation and the correction field

A central contribution of the theory is the decomposition of guided transport into a pretrained component and a correction component. If
\[
\pi_t(x)=\frac{1}{Z_t}p_t(x)\exp(-f_t(x)),
\]
then the pretrained velocity \(u_t\), which generates \(p_t\), need not generate \(\pi_t\). The paper defines
\[
g_t(x)=\partial_t f_t(x)+\nabla f_t(x)^\top u_t(x)
\]
and proves the residual identity
\[
\partial_t\pi_t+\nabla\cdot(\pi_t u_t)=-\pi_t\big(g_t-\mathbb E_{\pi_t}[g_t]\big).
\]
This equation gives an exact characterization of the mismatch between the guided marginal evolution and the pretrained transport field [2605.07950].

To restore a valid transport description of the guided path, the paper introduces a correction field \(w_t\) solving the Poisson-type equation
\[
\nabla\cdot(\pi_t w_t)=\pi_t\big(g_t-\mathbb E_{\pi_t}[g_t]\big).
\]
Then \(u_t+w_t\) is a transport velocity for \(\pi_t\):
\[
\partial_t\pi_t+\nabla\cdot(\pi_t(u_t+w_t))=0.
\]
In this decomposition, \(u_t\) is the pretrained marginal transport velocity and \(w_t\) is the additional velocity required to bend the pretrained path \(p_t\) into the guided path \(\pi_t\).

This distinction is what makes VA-SALD different from simply applying SALD to \(\pi_t\). Standard SALD on the guided path must track the full guided transport, whose complexity is governed by \(\mathcal A_\alpha(\pi,u+w)\). VA-SALD instead incorporates \(u_t\) directly in the sampler, so the convergence bounds depend only on \(\mathcal A_\alpha(\pi,w)\), the complexity of the correction field. When guidance is relatively mild, the paper states that \(\|w_t\|\ll \|u_t\|\) and typically \(\mathcal A_\alpha(\pi,w)\ll \mathcal A_\alpha(\pi,u+w)\). This suggests that slowdown can be concentrated on compensating for guidance-induced deviation rather than re-tracking the entire pretrained generative flow.

The unguided limit makes the decomposition transparent. If \(f_t\equiv 0\), then \(g_t\equiv 0\), the residual vanishes, and \(w_t\equiv 0\). In that regime, VA-SALD reduces to exact tracking of the pretrained marginal path, and the guidance-complexity term disappears from the theory.

## 4. KL theory, slowdown, and functional inequalities

The paper establishes non-asymptotic convergence guarantees through KL differential inequalities. For SALD, if each \(\pi_t\) satisfies a log-Sobolev inequality with constant \(\lambda_t\ge 0\), and \(v_t\) is a transport velocity for \(\pi_t\), the law \(\rho_S\) at terminal algorithmic time satisfies a bound of the form
\[
KL(\rho_S\|\pi_T)\le \text{contraction term on }KL(\rho_0\|\pi_0)+\text{path-complexity term involving }\mathfrak E_\alpha(\pi_t,v_t).
\]
For linear slowdown \(t(s)=s/r\), the bound simplifies to
\[
KL(\rho_S\|\pi_T)\le e^{-r\int_0^T\lambda_t dt}\,e^{T/(2r\alpha)}\,KL(\rho_0\|\pi_0)
+\frac{1}{2r}e^{T/(2r\alpha)}\,\mathcal A_\alpha(\pi,v).
\]
As \(r\to\infty\), the paper writes
\[
KL(\rho_S\|\pi_T)=O\!\left(e^{-r\int_0^T\lambda_tdt}\,KL(\rho_0\|\pi_0)+\frac{\mathcal A_0(\pi,v)}{r}\right).
\]
Two effects are explicit: slowdown improves contraction of initialization mismatch through accumulated intermediate log-Sobolev constants, and it reduces tracking error through the \(1/r\) scaling of path energy. The discrete-time SALD theory yields analogous rates and an iteration complexity \(K=O(\varepsilon^{-6})\) to reach \(O(\varepsilon^2)\) KL accuracy [2605.07950].

For VA-SALD, the general theory introduces a comparison field \(c_t\) and mismatch \(m_t=v_t-c_t\), where \(v_t\) is the true transport velocity of \(\pi_t\). The continuous-time KL bound has the form
\[
KL(\rho_S\|\pi_T)\le \text{contraction term depending on }\int_0^T \frac{\sigma_t^2}{2}\dot s(t)\lambda_t\,dt
+\text{mismatch-complexity term depending on }\mathfrak E_\alpha(\pi_t,m_t).
\]
In the actual diffusion-model specialization, \(c_t=u_t\), so \(m_t=w_t\). Under linear slowdown and with \(\sigma_{\min}=\min_t \sigma_t\), Corollary 2 gives
\[
\begin{aligned}
KL(\rho_S\|\pi_T)
&\le \exp\!\Big(-\frac{r\sigma_{\min}^2}{2}\int_0^T\lambda_t\,dt\Big)\exp\!\Big(\frac{T}{r\sigma_{\min}^2\alpha}\Big)KL(\rho_0\|\pi_0) \\
&\quad+\frac{1}{r\sigma_{\min}^2}\exp\!\Big(\frac{T}{r\sigma_{\min}^2\alpha}\Big)\mathcal A_\alpha(\pi,w).
\end{aligned}
\]
For small \(\alpha\), the asymptotic form is
\[
KL(\rho_S\|\pi_T)=O\!\Big(e^{-\frac{r\sigma_{\min}^2}{2}\int_0^T\lambda_tdt}\,KL(\rho_0\|\pi_0)+\frac{A_0(\pi,w)}{r}\Big).
\]
The interpretation is direct: guidance bias is controlled by the action of the correction field \(w_t\), not by the action of the full guided velocity \(u_t+w_t\). In the unguided case \(w_t\equiv 0\), the path-complexity term vanishes and only contraction of initialization mismatch remains.

The appendix further connects correction complexity to a Poincaré inequality. If \(\pi_t\) satisfies PI with constant \(\lambda_t>0\), then the solution \(w_t\) of the Poisson equation satisfies
\[
\|w_t\|_{L_2(\pi_t)}^2\le \frac{1}{\lambda_t}\,\mathrm{Var}_{\pi_t}[g_t],
\]
and therefore
\[
A_0(\pi,w)\le \int_0^T \frac{1}{\lambda_t}\mathrm{Var}_{\pi_t}[g_t]\,dt.
\]
This identifies guide-induced complexity with the variance of the directional time derivative \(g_t\), giving a functional-analytic interpretation of when training-free guidance is easy or hard.

## 5. Algorithms, implementations, and empirical results

The general Euler–Maruyama discretization of VA-SALD is
\[
X_{k+1}^\eta
=
X_k^\eta+\eta\left(\dot t_k\,c_{t_k}(X_k^\eta)+\frac{\sigma_{t_k}^2}{2}\nabla\log \pi_{t_k}(X_k^\eta)\right)+\sigma_{t_k}\sqrt{\eta}\,\xi_k,
\qquad \xi_k\sim \mathcal N(0,I_d),
\]
with \(c_t=u_t\) in the diffusion-model setting. For linear slowdown, \(t_k=k\eta/r\), and the conceptual Itô-diffusion implementation uses
\[
u_t(x)=-B_t(x)+\frac{\sigma_t^2}{2}\nabla\log p_t(x),
\]
together with the guide gradient \(\nabla f_t\). The algorithm samples \(X_0\sim \rho_0\), iterates the VA-SALD-EM update, and outputs \(X_K\) as an approximate sample from \(\pi_T\). The main hyperparameters are the slowdown factor \(r\), step size \(\eta\), guidance scale \(c\), and model noise schedule \(\sigma_t\). The paper states that larger \(r\) improves tracking and bias suppression at linear computational cost, and suggests choosing \(r\gtrsim A_0(\pi,w)/\varepsilon^2\) with \(\eta\lesssim \varepsilon^2/(r\bar\Delta)\) for discretization control [2605.07950].

A concrete large-scale implementation is given for flow-matching models, including experiments with Stable Diffusion 3.5 Medium. For flow matching, the forward ODE is
\[
d\boldsymbol x_\tau=v_\tau(\boldsymbol x_\tau)\,d\tau,
\]
and an SDE with the same marginals is
\[
d\boldsymbol x_\tau=\left(v_\tau(\boldsymbol x_\tau)+\frac{\sigma_\tau^2}{2}\nabla\log p_\tau(\boldsymbol x_\tau)\right)d\tau+\sigma_\tau\,dW_\tau,
\]
with score
\[
\nabla\log p_\tau(\boldsymbol x_\tau)= -\frac{\boldsymbol x_\tau}{\tau}-\frac{1-\tau}{\tau}v_\tau(\boldsymbol x_\tau).
\]
After reversing time and applying slowdown, the implementation uses a specialized VA-SALD SDE and its Euler discretization. For black-box guidance, \(\nabla f_t\) is approximated by zeroth-order gradient estimation,
\[
\nabla f_t(x)\approx \frac{1}{N\bar \sigma_t}\sum_{i=1}^N f(x+\bar \sigma_t\epsilon^i)\,\epsilon^i,\qquad \epsilon^i\sim \mathcal N(0,I),
\]
with \(\bar \sigma_t=\sigma_t\sqrt{\eta}\), together with group reward normalization.

The empirical study has two parts. In synthetic VP diffusion experiments, the paper considers a two-Gaussian data distribution with a two-moons guide, and an eight-Gaussian ring with a left-half mode penalty. SALD’s terminal KL to the guided target decreases monotonically with the slowdown factor \(r\), matching the theory. VA-SALD achieves lower terminal KL for smaller budgets, with flatter and better KL curves than SALD under matched budgets. DOIT, under matched proposal-particle budgets, remains significantly worse in terminal KL.

In guided image generation, the backbone is Stable Diffusion 3.5 Medium in flow-matching form, and the rewards include Aesthetic score, PickScore, and CLIPScore. Under matched numbers of steps and batch query size, and with guidance scale \(c=8\), VA-SALD consistently achieves higher reward and visually better images than the baselines FM-ZG and FM-Evolv. Those baselines often exhibit instability and artifacts at high guidance strength, whereas VA-SALD produces samples that evolve stably as \(r\) increases and remain high quality. The paper also reports qualitative examples for prompts such as “bear” and “wolf” across varying \(r\) and guidance strengths \(c=5,8\), showing gradual and stable guidance under VA-SALD [2605.07950].

## 6. Methodological position, misconceptions, and open problems

VA-SALD occupies a specific position within guided generative modeling. Relative to standard Langevin or ancestral sampling, it does not merely add a heuristic guide term to the pretrained sampler; it incorporates the pretrained transport velocity \(u_t\) and applies slowdown to a principled mismatch term. Relative to standard SALD on the guided path, it does not pay for the action of the full guided transport; its KL bounds depend only on the correction energy \(\mathcal A_\alpha(\pi,w)\). Relative to classifier or classifier-free guidance, it is framed through marginal KL control, intermediate functional inequalities, and path complexity rather than through heuristic score modification. Relative to Doob \(h\)-transform approaches such as DOIT and Doob’s matching, it does not promise exact terminal tilting, but it avoids training a time-dependent guide network or performing expensive on-the-fly simulations [2605.07950].

The relation to Doob \(h\)-transforms is especially instructive. The exact terminal tilt \(p_Te^{-f}\) can be realized through
\[
h_t(x)=\mathbb E[e^{-f(X_T)}\mid X_t=x],
\]
with a reverse-process drift involving \(\nabla\log p_t(x)+\nabla\log h_t(x)\). VA-SALD instead works with a prescribed guide schedule \(f_t\), uses only \(\nabla f_t\), and controls the resulting bias through slowdown and the correction complexity \(\mathcal A_\alpha(\pi,w)\). The paper explicitly characterizes this as a trade: no Doob optimality guarantee, but a training-free inference procedure with quantitative KL bounds.

Several limitations and open questions are highlighted. First, slowdown is computationally costly: increasing \(r\) improves tracking but linearly increases the number of iterations. Second, the choice of guide schedule \(f_t\) is unresolved; the correction complexity depends on \(\partial_t f_t\) and \(\nabla f_t^\top u_t\) through \(g_t\), so designing schedules that minimize \(\mathcal A_\alpha(\pi,w)\) remains open. Third, the theory assumes log-Sobolev or Poincaré inequalities, smoothness, and dissipativity conditions that are natural for many annealing paths and dissipative diffusions but are not trivial to verify for large-scale high-dimensional generative models. Fourth, extremely strong or adversarial guides may produce large \(A_0(\pi,w)\), requiring very large slowdown factors. Fifth, the empirical evaluation is illustrative rather than exhaustive: it covers two-dimensional synthetic experiments and a limited set of SD3.5 prompts rather than large prompt suites or human preference studies.

Taken together, these features define VA-SALD as a training-free guided sampling method whose novelty lies in its explicit separation of base transport from guidance correction. Its theory formalizes the intuition that pretrained generative models already encode a useful path of marginals \(p_t\), and that guidance should be treated as a controlled perturbation of that path rather than as an entirely new trajectory to be tracked from scratch [2605.07950].

Source: https://www.emergentmind.com/topics/velocity-aware-sald-va-sald