---
title: Conditional Backward-in-Time Score
url: https://www.emergentmind.com/topics/conditional-backward-in-time-score
type: topic
---

# Conditional Backward-in-Time Score

Conditional backward-in-time score denotes the time-indexed gradient of a conditional log-density that governs reverse-time denoising, posterior sampling, or conditioned generation in diffusion-type models. In continuous settings it is typically written as \(s(\mathbf{x}, t \mid \mathbf{y}) := \nabla_{\mathbf{x}} \log p_t(\mathbf{x}\mid \mathbf{y})\) or \(s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)\), and it replaces the marginal score in reverse-time dynamics so that trajectories initialized from a tractable reference distribution evolve toward conditional targets [2501.18913][2210.04872]. Related formulations appear in terminal-condition BSDE models, where conditioning is imposed through a backward boundary value, and in discrete-state diffusion models, where gradients are replaced by conditional log-odds and reverse jump rates [2304.13224][2211.16750].

## 1. Definition and validity conditions

In score-based diffusion, the unconditional score at diffusion time \(t\) is \(s(x,t):=\nabla_x \log p_t(x)\). The conditional backward-in-time score replaces this with the conditional object
\[
s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),
\]
or, in simulator-based Bayesian inference,
\[
s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).
\]
By Bayes’ rule, the static posterior score satisfies
\[
\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),
\]
and the diffusion-time version transports this decomposition to the perturbed family \(p_t(\cdot\mid x)\) [2210.04872].

A fundamental property of score functions is that they have zero mean under the target density:
\[
\mathbb{E}_{p_t(x\mid y)}\!\left[\nabla_x \log p_t(x\mid y)\right]=0.
\]
This identity supplies a strict validity criterion for any purported conditional score estimator. In particular, if an estimator has a large nonzero mean under \(p_t(\cdot\mid y)\), it cannot be a valid score for that conditional density [2501.18913].

## 2. Appearance in reverse-time dynamics

For a forward SDE
\[
d x_t = f(x_t,t)\,dt + g(t)\,dW_t,
\]
the reverse-time SDE conditioned on \(y\) is
\[
d x_t = \Big(f(x_t,t) - g(t)^2 \nabla_x \log p_t(x_t\mid y)\Big)\,dt + g(t)\,d\bar W_t,
\]
and the associated probability-flow ODE is
\[
\frac{d x_t}{dt}=f(x_t,t)-\tfrac{1}{2}g(t)^2 \nabla_x \log p_t(x_t\mid y).
\]
In both cases the conditional backward-in-time score is the term that determines the reverse drift [2501.18913][2210.04872].

Discrete-time DDPM parameterizations implement the same principle through conditional Gaussian reverse kernels. In CSDI, for example,
\[
p_\theta(x_{0:T}^{\mu}\mid x_0^o):=p(x_T^\mu)\prod_{t=1}^T p_\theta(x_{t-1}^\mu\mid x_t^\mu,x_0^o),
\]
with
\[
p_\theta(x_{t-1}^\mu\mid x_t^\mu,x_0^o):=\mathcal{N}(x_{t-1}^\mu;\mu_\theta(x_t^\mu,t\mid x_0^o),\sigma_\theta(x_t^\mu,t\mid x_0^o)I).
\]
Here the conditional denoiser \(\epsilon_\theta(x_t^\mu,t\mid x_0^o)\) is a rescaled conditional score model, and only the missing components are denoised while the observed entries remain fixed as conditioning inputs [2107.03502].

A different realization appears in BSDE-based diffusion. There the backward process is specified by
\[
dY_t=-\hat f(s_\theta(Y_t,t),Z_t)\,dt+Z_t\,dW_t,\qquad Y_T=\xi,
\]
or, equivalently,
\[
Y_t=\xi+\int_t^T \hat f(s_\theta(Y_s,s),Z_s)\,ds-\int_t^T Z_s\,dw_s.
\]
Conditioning is built into the terminal condition \(Y_T=\xi\): the score \(s_\theta\) is evaluated along trajectories constrained to reach the prescribed terminal distribution. Under a Lipschitz generator and square-integrability, the BSDE has a unique adapted solution \((Y_t,Z_t)\) [2304.13224].

## 3. Estimation strategies and diagnostic criteria

Conditional backward-in-time scores are usually learned by denoising score matching or by equivalent \(\epsilon\)-prediction objectives. In SNPSE, the amortized posterior-score objective is
\[
J_{\mathrm{post}}(\psi_{\mathrm{post}})=\tfrac{1}{2}\int_0^T \lambda_t\,
\mathbb{E}\!\left[\left\|s_{\psi_{\mathrm{post}}}(\theta_t,x,t)-\nabla_{\theta_t}\log p_{t\mid 0}(\theta_t\mid \theta_0)\right\|^2\right]dt,
\]
which directly targets \(\nabla_\theta \log p_t(\theta\mid x)\) using only prior sampling and simulator calls [2210.04872]. In CSDI, the conditional denoising objective is
\[
\min_\theta \mathbb{E}\left[\|\epsilon-\epsilon_\theta(x_t^\mu,t\mid x_0^o)\|_2^2\right],
\qquad
x_t^\mu=\sqrt{\alpha_t}x_0^\mu+(1-\alpha_t)\epsilon,
\]
with masking used so that the loss is applied only on imputation targets [2107.03502].

A central diagnostic question is whether a practical guidance rule actually estimates a conditional score. “Rethinking Diffusion Posterior Sampling” reports that the conditional score approximation employed by DPS “is not as effective as previously assumed, but rather aligns more closely with the principle of maximizing a posterior.” On 512×512 ImageNet images, the paper reports that DPS’s estimated conditional score “significantly diverges from the score of a well-trained conditional diffusion model,” that “the mean of DPS’s conditional score estimation deviates significantly from zero,” and that DPS “generates high-quality samples with significantly lower diversity” [2501.18913].

The same analysis gives concrete zero-mean diagnostics. At \(X_T\) over 1000 samples, the reported mean magnitudes are approximately \(0.4026\) for the unconditional score, \(0.3939\) for StableSR, and \(5.8568\) for DPS with \(\zeta_t=4.8\). For \(k=50\) posterior samples, per-pixel standard deviation is \(0.0453\) for DPS with \(\zeta_t=4.8\) and \(0.3939\) for StableSR. On ImageNet 512×512 for SR×8, the same comparison reports LPIPS \(0.4137\) and FID \(58.48\) for DPS, versus LPIPS \(0.2855\) and FID \(29.12\) for StableSR [2501.18913].

## 4. Domain-specific realizations

In simulation-based inference, the conditional backward-in-time score is the posterior score of perturbed parameters, and reverse-time diffusion is used to turn tractable reference noise into posterior samples. SNPSE introduces Sequential Neural Posterior Score Estimation, embeds the score model into a sequential proposal scheme, and validates it on Gaussian Mixture, Two Moons, Gaussian Linear Uniform (10D), and SLCP. The reported evaluation metric is C2ST, where lower is better and \(0.5\) indicates perfect posterior estimation. The paper states that NPSE and NLSE are “often comparable or superior to state-of-the-art sequential methods” such as SNPE, SNLE, and SNRE, especially at small budgets around 1000 simulations [2210.04872].

In probabilistic time-series imputation, the conditioning variable is the observed subseries. CSDI defines the reverse process directly for the missing values conditioned on observed values, masks, timestamps, and feature embeddings, and uses 2D attention across temporal and feature axes. The paper reports that CSDI “improves by 40-65% over existing probabilistic imputation methods on popular performance metrics,” and that deterministic imputation “reduces the error by 5-20% compared to the state-of-the-art deterministic imputation methods.” For CRPS on healthcare data with \(10/50/90\%\) missingness, the reported values are \(0.238/0.330/0.522\) for CSDI, versus \(0.360/0.458/0.671\) for unconditional diffusion; for MAE the reported CSDI values are \(0.217/0.301/0.481\) [2107.03502].

In inverse physics, conditioning is imposed by fixing the terminal state and integrating backward through an approximate inverse simulator plus a learned score correction. “Solving Inverse Physics Problems with Score Matching” uses the update
\[
\mathbf{x}_{t_m}\approx \mathbf{x}_{t_{m+1}}+\Delta t\left[\widetilde{\mathcal{P}^{-1}}(\mathbf{x}_{t_{m+1}})+s_\theta(\mathbf{x}_{t_{m+1}},t_{m+1})\right],
\]
and shows that single-step training is equivalent to score matching, while multi-step rollout training relates to maximum-likelihood training of a corresponding probability flow. The reverse-time SDE version provides posterior sampling, whereas the probability-flow ODE yields a deterministic ML trajectory [2301.10250].

## 5. Discrete and categorical analogues

For categorical data, the gradient \(\nabla_x \log p_t(x)\) is not properly defined, so the conditional backward-in-time score is replaced by conditional marginals and reverse jump intensities. In score-based continuous-time discrete diffusion, the reverse-time CTMC has rate matrix
\[
R_t(x,y)=\frac{q_t(y)}{q_t(x)}Q_t(y,x),
\]
and under single-coordinate jumps the learned reverse rate is
\[
R_t^d(x_t,y;\theta)=
\frac{p_t(X_t^d=y^d\mid x_t^{\setminus d};\theta)}
{p_t(X_t^d=x_t^d\mid x_t^{\setminus d};\theta)}
Q_t(\{y,x_t\}).
\]
The log-ratio
\[
\log p_t(X_t^d=y^d\mid x_t^{\setminus d};\theta)-
\log p_t(X_t^d=x_t^d\mid x_t^{\setminus d};\theta)
\]
plays the role of a conditional backward-in-time “score” analogue [2211.16750].

Learning proceeds by matching singleton conditional marginals. The paper states that matching \(\pi(X^d=c\mid x^{\setminus d})\) for all \(d\) and \(x\) is sufficient to match the joint distribution, and proposes the simplified objective
\[
\theta^*=\arg\min_\theta \int_0^T \sum_{x_t\in\mathcal{X}} q_t(x_t)
\left[\sum_{d=1}^D -\log p_t(X^d=x_t^d\mid x_t^{\setminus d};\theta)\right]dt.
\]
An implicit parameterization through \(p_{0\mid t}(X_0^d\mid x_t^{\setminus d};\theta)\) then yields analytical reverse steps [2211.16750].

Empirically, the same framework reports strong results on both synthetic and real-world benchmarks. On CIFAR-10 in VQ code space, SDDM-VQ achieves IS \(=8.98\) and FID \(=12.23\), improving over D3PM-VQ with IS \(=8.85\) and FID \(=16.47\). At 50 reverse steps, the analytical sampler reports FID \(=14.99\), whereas D3PM reports FID \(=84.72\). On monophonic music, SDDM reports Hellinger Distance \(=0.3736\pm 0.0024\) and Proportion of Outliers \(=0.1093\pm 0.0016\) [2211.16750].

## 6. Exact conditional identities, variance reduction, and matrix gating

Several recent works replace approximate conditional-score surrogates with exact or variance-controlled identities. For a Gaussian mixture prior
\[
p_X(x)=\sum_{k=1}^K \pi_k \,\phi(x;\mu_k,\Sigma_k),
\]
and linear-Gaussian observations \(Y=HX+\epsilon_y\), \(\epsilon_y\sim\mathcal{N}(0,\Sigma_Y)\), “Exact Conditional Score-Guided Generative Modeling for Amortized Inference in Uncertainty Quantification” derives a closed-form conditional score
\[
S_{Z_t\mid Y}(z_t\mid y)=\nabla_{z_t}\log p_{Z_t\mid Y}(z_t\mid y)
\]
as a mixture of componentwise scores and likelihood corrections weighted by exact responsibilities \(p(\xi=k\mid Z_t=z_t,Y=y)\). The reverse-time probability-flow ODE
\[
dZ_t=\big[b(t)Z_t-\tfrac{1}{2}\sigma^2(t)S_{Z_t\mid Y}(Z_t\mid y)\big]dt,\qquad Z_1\sim\mathcal{N}(0,I),
\]
is then used as a training-free conditional sampler. The same paper uses the resulting noise-labeled pairs \((\epsilon,y,u)\) to train a feedforward network \(F_\theta(y,\epsilon)\) for amortized conditional inference. Reported numerical results include KL divergence “\(\approx 10^{-3}\)” in a 1D bimodal conditional example and projection KLs between \(0.013\) and \(0.052\) in a 20D two-mode GMM example [2506.18227].

For affine diffusions, “Variance-Reduced Diffusion Sampling via Conditional Score Expectation Identity” proves the exact Conditional Score Expectation identity
\[
s_t(y)=\Phi(t,0)^{-\top}\,\mathbb{E}[s_0(X_0)\mid X_t=y].
\]
This expresses the time-\(t\) score as a conditional expectation of the initial score under the forward dynamics. The paper then constructs a Self-Normalized Importance Sampling estimator and blends it with a Tweedie estimator through a state–time dependent convex combination. Reported examples include a 24D Navier–Stokes posterior, where MMD to MALA is \(0.126\) for Tweedie and \(0.090\) for the blend, with KSD \(15.8\) versus \(2.01\); and an MNIST deblurring posterior in a 15D PCA space, where PSNR improves from \(26.98\) dB to \(28.02\) dB and posterior coverage from \(92.6\%\) to \(100\%\) [2601.01594].

“Laplace–Fisher Gate Identities for Optimal Matrix-Gated Blended Score Estimation” further generalizes scalar blending to matrix-valued gates for OU diffusion reversal. With \(\alpha_t=e^{-t}\), \(\gamma_t=1-e^{-2t}\), and \(H_0=-\nabla^2\log p_0\), the variance-optimal gate is
\[
G_*(y,t)=\alpha_t^2\bigl(\alpha_t^2 I_d+\gamma_t\,\mathbb{E}[H_0(X_0)\mid Y_t=y]\bigr)^{-1}.
\]
Because the Tweedie–TSI disagreement has conditional mean zero, the gate changes variance without changing expected value. The paper states that LFGI improves posterior-density calibration and sampling diagnostics relative to the other tested score-estimator classes, and that known-evidence experiments check absolute calibration in Gaussian and non-Gaussian settings [2606.25169].

A plausible implication is that the recent literature is moving from heuristic conditional guidance toward three increasingly stringent regimes: direct conditional score learning, exact conditional score construction under tractable model classes, and variance-optimized blending identities for reverse-time sampling. Across these regimes, the defining invariant remains the same: the conditional backward-in-time score is the object that makes reverse dynamics target a conditional law rather than an unconditional marginal.

Source: https://www.emergentmind.com/topics/conditional-backward-in-time-score