---
title: Adjoint Matching & Sampling Methods
url: https://www.emergentmind.com/topics/adjoint-matching-sampling
type: topic
---

# Adjoint Matching & Sampling Methods

Searching arXiv for recent papers on adjoint matching and adjoint sampling.
Adjoint Matching and Adjoint Sampling denote a family of matching-based methods for fine-tuning flow or diffusion generators and for sampling from unnormalized densities by learning a controlled stochastic process whose terminal law realizes an exponential tilt of a base distribution or matches a target Boltzmann density. In the recent literature, the framework is formulated as stochastic optimal control (SOC) or Schrödinger bridge (SB) learning, and its defining computational move is to replace unstable backpropagation through multi-step denoising trajectories, or high-variance importance weighting, with regression onto an adjoint-derived target or an equivalent bridge-derived target [2409.08861][2504.11713][2605.00229].

## 1. Problem class and target distributions

A canonical formulation is the exponential tilt
\[
p^*(x)\propto p_{\text{base}}(x)\exp(r(x)),
\]
which subsumes both reward fine-tuning of a pretrained generative model and sampling from an unnormalized target density. In the unified treatment, unnormalized density sampling is recovered by taking \(p_{\text{base}}=\mathcal N(0,\sigma_0^2 I)\) and \(r(x)=-E(x)+\text{const}\), while reward fine-tuning uses a pretrained flow or diffusion model as \(p_{\text{base}}\) and a reward model \(r\) as the tilting term [2605.00229].

The controlled generative dynamics are typically written as
\[
dX_t^u = \bigl(b_0(X_t^u,t)+\sigma(t)u(X_t^u,t)\bigr)\,dt+\sigma(t)\,dB_t,
\]
with a quadratic control-energy penalty. In this representation, \(b_0\) is the base drift inherited from the reference sampler, \(u\) is the learned control, and the SOC objective is chosen so that the optimal terminal law coincides with the tilted target [2605.00229][2409.08861]. In energy-based sampling, the same structure appears as learning a controlled diffusion whose terminal marginal matches a Boltzmann distribution known only through an energy \(E(x)\) or an unnormalized density \(\nu(x)\propto e^{-U_1(x)}\) [2504.11713][2506.18165].

This common target class explains why “adjoint matching” and “adjoint sampling” now appear across image alignment, molecular Boltzmann sampling, diffusion-policy reinforcement learning, and flow-model post-training. The objective is not merely to increase reward or reduce energy pointwise; it is to learn a full path-space control whose endpoint distribution is the KL-regularized optimum associated with the base model [2605.10759][2601.14234].

## 2. Memoryless SOC and the original adjoint-matching formulation

The original memoryless SOC analysis established that naïvely fine-tuning a dynamical generator with terminal reward \(r(X_1)\) is biased because the optimal path law acquires an extra factor depending on the initial noise, written in the paper as \(V(X_0,0)\). As a result, the desired marginal tilt \(p^{\text{base}}(X_1)\exp(r(X_1))\) is recovered only when \(X_0\) and \(X_1\) are independent under the base process, i.e. when the fine-tuning dynamics are memoryless [2409.08861]. Within the family
\[
dX_t=b(X_t,t)\,dt+\sigma(t)\,dB_t,
\]
the same work proves that, in order to allow arbitrary noise schedules and still generate samples according to the tilted distribution, the fine-tuning problem must be done with the memoryless noise schedule \(\sigma(t)=\sqrt{2\eta_t}\) [2409.08861].

Adjoint Matching turns the SOC optimum into a regression problem. In the unified SOC presentation, the loss is
\[
\mathcal L_{\text{Adj-Match}}(u;X^u)
:= \mathbb E\!\left[\int_0^1 \left\|u(X_t^u,t)+\sigma(t)\,\tilde a(t;X^u)\right\|^2 dt\right],
\]
where the “lean adjoint” satisfies
\[
d\tilde a(t;X^u)= -\nabla_x b_u(X_t^u,t)^{\!\top}\,\tilde a(t;X^u)\,dt,
\qquad
\tilde a(1;X^u)=-\nabla_x r(X_1^u).
\]
The adjoint is “lean” because it removes terms that are zero at the optimum, and the stated consequence is that if \(u\) minimizes the expected loss and the loss is driven to zero, then \(u\) is the optimal control \(u^*\) [2605.00229]. In the original memoryless fine-tuning paper, this same idea appears as a regression objective using a simplified backward adjoint ODE rather than importance sampling or full adjoint differentiation through the optimized trajectory [2409.08861].

A recurrent point of clarification is that Adjoint Matching is a training objective, not a different inference-time sampler. In later continuous-action reinforcement learning work, this distinction is made explicit: adjoint matching changes how the flow is trained, while action sampling still proceeds by integrating the learned flow or SDE from noise to action [2601.14234].

## 3. Adjoint Sampling, reciprocal regression, and fixed-point bridge matching

Adjoint Sampling specializes the framework to unnormalized density sampling with a zero-drift reference diffusion,
\[
dX_t=\sigma(t)\,dB_t,\qquad X_0=0.
\]
In this case the lean adjoint is constant in time, and the matching loss simplifies to regressing the drift against the terminal energy-gradient correction. The decisive computational step is reciprocal adjoint matching: instead of simulating full controlled trajectories for every gradient update, one samples \(X_1\sim p_1^{\bar u}\), then samples \(X_t\sim p^{\text{base}}_{t|1}(\cdot\mid X_1)\), and optimizes
\[
\mathcal L_{\text{RAM}}(u)
=
\int_0^1 \lambda(t)\,
\mathbb E_{X_t\sim p^{\text{base}}_{t|1},\,X_1\sim p_1^{\bar u}}
\left[
\frac12\|u(X_t,t)+\sigma(t)\nabla g(X_1)\|^2
\right]dt.
\]
Because the base posterior \(p^{\text{base}}_{t|1}\) is available in closed form, replay-buffer reuse makes it possible to perform many inner optimization steps per expensive model sample or energy-gradient evaluation. The paper states this as the first on-policy approach that allows significantly more gradient updates than the number of energy evaluations and model samples [2504.11713].

Bridge Matching Sampler (BMS) recasts this family more generally as a fixed-point diffusion-matching method rooted in Nelson’s relation. The target path law is expressed as
\[
\Pi^*=\Pi^*_{0,T}P_{|0,T},
\]
with a reciprocal or bridge measure defined from an endpoint coupling and a tractable reference bridge. The corresponding optimal Markov drift is the Markovian projection
\[
u^*(x,t)=\mathbb E_{\Pi^*}\!\left[\xi(X,t)\mid X_t=x\right],
\]
which induces the fixed-point map
\[
u_{i+1}=\Phi(u_i),\qquad
\Phi(u_i):=\mathbb E_{\Pi^i}\!\left[\xi(X,t)\mid X_t\right].
\]
The same paper shows that previous adjoint and matching methods arise as special cases of the general pathwise target drift, and that the independent coupling \(\Pi^*_{0,T}=p_{\mathrm{prior}}\otimes p_{\mathrm{target}}\) yields a single fixed-point regression objective rather than alternating updates between drift and terminal potential [2603.00530].

BMS also introduces a damped update
\[
u_{i+1}=\alpha\,\Phi(u_i)+(1-\alpha)u_i,
\qquad \alpha\in(0,1],
\]
and proves that it is equivalent to a proximal or regularized matching loss that keeps updates close to the previous iterate. The interpolation coefficient \(c(t)\) is further interpreted as a control variate, and a learnable boundary-safe parameterization \(c^\phi(t)=\gamma(t)+\gamma(t)(1-\gamma(t))\,\mathrm{NN}^\phi(t)\) is proposed to reduce conditional variance [2603.00530]. This places adjoint matching and adjoint sampling inside a broader fixed-point and bridge-regression viewpoint.

## 4. Schrödinger bridges, annealing, and neighboring matching formulations

Adjoint Schrödinger Bridge Sampler (ASBS) generalizes Adjoint Sampling beyond the restrictive memoryless or Dirac-source setting. Its core claim is that Adjoint Sampling corresponds to a Schrödinger half-bridge special case, whereas ASBS learns the full SB with arbitrary source distribution \(\mu\) by alternating Adjoint Matching and Corrector Matching. In the SOC view, the SB problem is equivalent to an SOC objective with terminal cost \(\log \frac{\varphi_1}{\nu}\), and the paper states a global convergence theorem: the alternating optimization procedure converges to the Schrödinger bridge solution \(u^\star\), provided each matching stage reaches its critical point [2506.22565].

Adjoint Schrödinger Bridge Matching (ASBM) pushes the same line in a non-memoryless generative regime. It learns a forward SB dynamic as a coupling-construction problem from data to an energy-defined prior, then trains the backward generative dynamic with a bridge-matching loss under the induced coupling. The paper’s central claim is that, by operating outside the memoryless regime, ASBM yields more informative endpoint couplings, straighter trajectories, and better low-NFE generation than memoryless diffusion or prior SB baselines [2602.15396].

Non-equilibrium Annealed Adjoint Sampler (NAAS) changes the reference process rather than the optimization principle. Instead of a canonical Brownian reference, it uses annealed reference dynamics
\[
dX_t = -\frac{\sigma_t^2}{2}\nabla U_t(X_t)\,dt + \sigma_t\, dW_t,
\]
so that the reference already guides samples toward high-density regions of the target. Training still proceeds through a lean adjoint system and a regression loss of the form
\[
\mathcal{L}(u_\theta;X)=\int_0^1 \|u_\theta(X_t,t)+\sigma_t a(t;X)\|^2\,dt,
\]
but the method avoids the importance-sampling variance that affects many annealed samplers [2506.18165].

Tilt Matching is a closely related, though differently derived, matching method based on stochastic interpolants. It studies the velocity field that transports \(\rho_0\) to a reward-tilted terminal law \(\rho_{1,a}\propto \rho_1 e^{a r}\), and derives the “Covariance ODE”
\[
\frac{\partial b_{t,a}(x)}{\partial a}
=
\operatorname{Cov}_a\!\left(\dot I_t^a,\; r(x_1^a)\mid I_t^a=x\right).
\]
Its practical importance for the adjoint literature is negative as much as positive: it shows that reward-aware matching can be done using only scalar rewards, without reward gradients, backpropagation through trajectories, backward adjoint SDEs, HJB equations, or boundary-value control formulations [2512.21829]. This suggests a broader methodological spectrum in which adjoint matching is one particularly structured member.

## 5. Discrete spaces, function spaces, and other meanings of “adjoint matching”

Discrete Adjoint Matching (DAM) extends the continuous AM idea to discrete generative models represented as continuous-time Markov chains. In this setting the control object is a rate \(u_t(y,x)\), not a drift, and the optimal controlled rate has multiplicative form
\[
u^\star_t(y,x)=u^{\text{base}}_t(y,x)\,e^{-V_t(y)+V_t(x)}.
\]
DAM introduces a discrete adjoint whose terminal condition is
\[
\tilde a_1(y;\cdot)=e^{-g(y)+g(X_1)},
\]
and uses generalized KL divergence as the natural matching objective for nonnegative rates. The paper emphasizes that DAM is not a direct gradient analogue of continuous AM; it is a statistical estimator of a multiplicative exponential value ratio [2602.07132].

Discrete Adjoint Schrödinger Bridge Sampler develops a related extension from discrete SB theory. It argues that the core mechanism of AM is state-space agnostic, but identifies a necessary cyclic group structure on the state space so that additive-noise target-matching identities remain available. Under that structure, controller and corrector ratios can be trained by Bregman-regression objectives, and the resulting discrete ASBS is reported to be competitive in sample quality with advantages in training efficiency and scalability [2602.08243].

Functional Adjoint Sampler (FAS) moves in the opposite direction, generalizing Adjoint Sampling from \(\mathbb R^d\) to infinite-dimensional Hilbert spaces. The target becomes a Gibbs-type measure
\[
\pi(x)=\frac{1}{Z}e^{-U(x)}\nu(x)
\]
on a separable real Hilbert space, and the adjoint characterization is derived with the stochastic maximum principle. The training objective takes the form
\[
\mathcal L_{\mathrm{FAS}}(\theta)
=
\int_0^T
\mathbb E_{P_{t|T}P_T^{\bar\alpha}}
\left[
\frac12
\left\|
\alpha^\theta(X_t,t)+\sigma_t Q^{1/2}e^{-(T-t)\mathcal A^\dagger}Y_T
\right\|_{\mathcal H}^2
\right]dt,
\]
with applications to transition-path sampling under exact endpoint constraints [2511.06239].

The phrase “adjoint matching” also has earlier and broader uses outside modern diffusion-model training. In Monte Carlo transport, adjoint methods were used as importance-sampling accelerators, and hybrid methods constructed unbiased estimators by combining a partial deterministic adjoint solve with analog or heuristic Monte Carlo sampling [1104.2550]. In 4D-Var data assimilation, adjoint-matching neural surrogates were trained so that their derivative structure, or adjoint-vector products, matched a high-fidelity dynamical model, because the assimilation optimizer depends on both forward and adjoint dynamics [2111.08626]. These usages share the adjoint-derived supervision idea but differ technically from generative SOC matching.

## 6. Reinforcement learning and preference-alignment variants

In continuous-action reinforcement learning, Q-learning with Adjoint Matching (QAM) uses adjoint matching to optimize expressive flow or diffusion policies with respect to a learned critic without backpropagating through the full denoising chain. The actor is trained by minimizing
\[
L_{\mathrm{AM}}(\theta)
=
\mathbb E_{s\sim D,\{a_t\}_t}
\left[
\int_0^1
\left\|
\frac{2(f_\theta(s,a_t,t)-f_\beta(s,a_t,t))}{\sigma_t}+\sigma_t\tilde g_t
\right\|_2^2 dt
\right],
\]
where the terminal condition is \(\tilde g_1=-\tau \nabla_{a_1}Q_\phi(s,a_1)\) and the backward “lean adjoint” is propagated under the fixed behavior flow \(f_\beta\). The stated benefit is an unbiased, expressive policy at the optimum, together with much greater optimization stability than direct backpropagation through the optimized flow [2601.14234].

Trust Region Q-Adjoint Matching (TRQAM) addresses the critic-error amplification problem in QAM by internalizing a trust-region parameter \(\lambda\) into the SOC dynamics. Its key theorem gives a closed-form expression for the path-space KL,
\[
D_{\mathrm{KL}}(\mathbb P^u\|\mathbb P^{\mathrm{base}})
=
\mathbb E_{\mathbf X\sim\mathbb P^u}
\left[
\frac{1}{2\lambda}\int_0^1 \|u(X_\tau,\tau)\|^2\,d\tau
\right],
\]
which enables projected dual descent on \(\lambda\) to control the exact deviation from the pretrained flow policy [2605.27079].

For flow-model preference alignment, a deterministic control pipeline has emerged. “Improved techniques for fine-tuning flow models via adjoint matching” formulates alignment as deterministic optimal control over velocity fields, introduces a truncated adjoint scheme focused on the terminal portion of the trajectory, and generalizes beyond quadratic KL-style regularization to any increasing, differentiable, strictly convex penalty \(f\). The practical loss regresses the learned velocity correction toward the adjoint-induced optimum over the retained final steps, which the paper reports as substantially cheaper than full-horizon adjoint matching [2605.06583].

Efficient Adjoint Matching (EAM) makes a different efficiency trade. It replaces the non-trivial pretrained base drift with a linear base drift \(b(x,t)=D(t)x\), modifies the terminal cost accordingly, and obtains a closed-form adjoint
\[
a(t;X_t)=\frac{(2C-1)t}{2Ct^2-2t+1}\,a(1;X_1).
\]
The reported consequences are that backward adjoint simulation disappears, training-time sampling can use a few-step deterministic ODE solver, and the method converges up to \(4\times\) faster than AM on text-to-image reward fine-tuning benchmarks [2605.11480].

Reinforce Adjoint Matching (RAM) goes further by preserving the pretraining regression template itself. Under KL-regularized reward maximization, it uses a REINFORCE identity and the memoryless noising law to derive a consistency loss in which one samples a clean endpoint from the current model, evaluates its reward, noises it analytically as in pretraining, and regresses against a reward-corrected target. The paper stresses that no SDE rollouts, backward adjoint sweeps, or reward gradients are required, and reports peak reward matching Flow-GRPO in up to \(50\times\) fewer training steps [2605.10759].

For online RL with diffusion policies, “Scalable Maximum Entropy Reinforcement Learning for Diffusion Policies via Adjoint Matching” adopts reciprocal adjoint matching under a memoryless Brownian reference, derives a simulation-free actor update from the terminal gradient of \(\log \Pi_1^0 + Q/\alpha\), and then simplifies the Jacobian term with an error-function squashing map. A trust-region penalty is added to stabilize the policy updates while preserving the fixed point of the unconstrained adjoint-matching objective [2606.22630].

## 7. Empirical behavior, misconceptions, and open questions

A broad empirical pattern is that adjoint-based matching remains attractive because it combines matching-style scalability with finite-variance training signals. The unified analysis of diffusion and flow post-training proves bias-variance decompositions in which Adjoint Matching and Adjoint Sampling have finite gradient variance, whereas Target Score Matching and Conditional Score Matching do not under the natural weighting. The same paper also derives norm bounds on the lean adjoint ODE as a theoretical explanation for the practical stability of adjoint-based methods [2605.00229].

In unnormalized density sampling, BMS is presented as enabling sampling at unprecedented scales while preserving mode diversity, with Gaussian-mixture experiments reaching \(d=2500\) and molecular results showing that damping is crucial because undamped ASBS and BMS can diverge or collapse [2603.00530]. Adjoint Sampling is reported to scale from classical energy functions to amortized conformer generation, with reciprocal projection improving recall, precision, and transfer [2504.11713]. NAAS likewise reports strong performance on many-well, funnel, Gaussian-mixture, mixture-of-Student’s-\(t\), and alanine-dipeptide benchmarks, with the annealed reference already providing competitive initial sample quality before learning [2506.18165].

In generative modeling, ASBM reports straighter trajectories and better low-NFE fidelity than memoryless diffusion and earlier SB baselines, and the deterministic flow-alignment papers report improved reward metrics together with better diversity and mode preservation than direct reward-backprop methods [2602.15396][2605.06583]. In discrete and functional settings, the reported gains include synthetic and mathematical reasoning tasks for DAM and exact-endpoint, high-transition-hit-rate transition-path sampling for FAS [2602.07132][2511.06239].

Several misconceptions recur in the literature. One is that adjoint methods necessarily require target samples; in fact, Adjoint Sampling, ASBS, BMS, and NAAS all target settings where only energies, rewards, or pointwise density evaluations are available [2504.11713][2506.22565][2603.00530][2506.18165]. Another is that adjoint methods always change the sampling procedure itself; QAM explicitly states that adjoint matching primarily changes the training objective, not the basic flow-sampling procedure [2601.14234].

Open issues are also stated directly. BMS notes that convergence theory and optimal damping schedules remain open [2603.00530]. QAM is guaranteed only for policies supported by the behavior prior and may struggle under severe support mismatch [2601.14234]. RAM attains scalability by dropping the path-cost correction in the exact adjoint decomposition, which the paper identifies as a deliberate approximation [2605.10759]. Discrete variants require nontrivial structural assumptions, such as cyclic group structure for target-matching identities in discrete ASBS [2602.08243]. These caveats indicate that adjoint matching and adjoint sampling are not a single algorithm but a growing design space whose common core is adjoint-derived regression for path-space control.

Source: https://www.emergentmind.com/topics/adjoint-matching-sampling