Conditional Backward-in-Time Score

• Conditional backward-in-time score is defined as the gradient of a conditional log-density that steers reverse dynamics in diffusion models.
• It replaces the marginal score in reverse-time SDEs and ODEs, enabling targeted generative sampling and robust Bayesian inference.
• Estimation strategies such as denoising score matching and variance-reduced techniques enhance its accuracy across both continuous and discrete model settings.

Conditional backward-in-time score denotes the time-indexed gradient of a conditional log-density that governs reverse-time denoising, posterior sampling, or conditioned generation in diffusion-type models. In continuous settings it is typically written as $$s(\mathbf{x}, t \mid \mathbf{y}) := \nabla_{\mathbf{x}} \log p_t(\mathbf{x}\mid \mathbf{y})$$ or $$s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)$$, and it replaces the marginal score in reverse-time dynamics so that trajectories initialized from a tractable reference distribution evolve toward conditional targets (Xu et al., 31 Jan 2025, Sharrock et al., 2022). Related formulations appear in terminal-condition BSDE models, where conditioning is imposed through a backward boundary value, and in discrete-state diffusion models, where gradients are replaced by conditional log-odds and reverse jump rates (Wang, 2023, Sun et al., 2022).

1. Definition and validity conditions

In score-based diffusion, the unconditional score at diffusion time $t$ is $s(x,t):=\nabla_x \log p_t(x)$. The conditional backward-in-time score replaces this with the conditional object

$$s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),$$

or, in simulator-based Bayesian inference,

$$s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).$$

By Bayes’ rule, the static posterior score satisfies

$$\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),$$

and the diffusion-time version transports this decomposition to the perturbed family $p_t(\cdot\mid x)$ (Sharrock et al., 2022).

A fundamental property of score functions is that they have zero mean under the target density: $$\mathbb{E}_{p_t(x\mid y)}\!\left[\nabla_x \log p_t(x\mid y)\right]=0.$$ This identity supplies a strict validity criterion for any purported conditional score estimator. In particular, if an estimator has a large nonzero mean under $p_t(\cdot\mid y)$, it cannot be a valid score for that conditional density (Xu et al., 31 Jan 2025).

2. Appearance in reverse-time dynamics

For a forward SDE

$$s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)$$0

the reverse-time SDE conditioned on $$s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)$$1 is

$$s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)$$2

and the associated probability-flow ODE is

$$s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)$$3

In both cases the conditional backward-in-time score is the term that determines the reverse drift (Xu et al., 31 Jan 2025, Sharrock et al., 2022).

Discrete-time DDPM parameterizations implement the same principle through conditional Gaussian reverse kernels. In CSDI, for example,

$$s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)$$4

with

$$s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)$$5

Here the conditional denoiser $$s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)$$6 is a rescaled conditional score model, and only the missing components are denoised while the observed entries remain fixed as conditioning inputs (Tashiro et al., 2021).

A different realization appears in BSDE-based diffusion. There the backward process is specified by

$$s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)$$7

or, equivalently,

$$s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)$$8

Conditioning is built into the terminal condition $$s(\theta, t \mid x) := \nabla_\theta \log p_t(\theta\mid x)$$9: the score $t$0 is evaluated along trajectories constrained to reach the prescribed terminal distribution. Under a Lipschitz generator and square-integrability, the BSDE has a unique adapted solution $t$1 (Wang, 2023).

3. Estimation strategies and diagnostic criteria

Conditional backward-in-time scores are usually learned by denoising score matching or by equivalent $t$2-prediction objectives. In SNPSE, the amortized posterior-score objective is

$t$3

which directly targets $t$4 using only prior sampling and simulator calls (Sharrock et al., 2022). In CSDI, the conditional denoising objective is

$t$5

with masking used so that the loss is applied only on imputation targets (Tashiro et al., 2021).

A central diagnostic question is whether a practical guidance rule actually estimates a conditional score. “Rethinking Diffusion Posterior Sampling” reports that the conditional score approximation employed by DPS “is not as effective as previously assumed, but rather aligns more closely with the principle of maximizing a posterior.” On 512×512 ImageNet images, the paper reports that DPS’s estimated conditional score “significantly diverges from the score of a well-trained conditional diffusion model,” that “the mean of DPS’s conditional score estimation deviates significantly from zero,” and that DPS “generates high-quality samples with significantly lower diversity” (Xu et al., 31 Jan 2025).

The same analysis gives concrete zero-mean diagnostics. At $t$6 over 1000 samples, the reported mean magnitudes are approximately $t$7 for the unconditional score, $t$8 for StableSR, and $t$9 for DPS with $s(x,t):=\nabla_x \log p_t(x)$0. For $s(x,t):=\nabla_x \log p_t(x)$1 posterior samples, per-pixel standard deviation is $s(x,t):=\nabla_x \log p_t(x)$2 for DPS with $s(x,t):=\nabla_x \log p_t(x)$3 and $s(x,t):=\nabla_x \log p_t(x)$4 for StableSR. On ImageNet 512×512 for SR×8, the same comparison reports LPIPS $s(x,t):=\nabla_x \log p_t(x)$5 and FID $s(x,t):=\nabla_x \log p_t(x)$6 for DPS, versus LPIPS $s(x,t):=\nabla_x \log p_t(x)$7 and FID $s(x,t):=\nabla_x \log p_t(x)$8 for StableSR (Xu et al., 31 Jan 2025).

4. Domain-specific realizations

In simulation-based inference, the conditional backward-in-time score is the posterior score of perturbed parameters, and reverse-time diffusion is used to turn tractable reference noise into posterior samples. SNPSE introduces Sequential Neural Posterior Score Estimation, embeds the score model into a sequential proposal scheme, and validates it on Gaussian Mixture, Two Moons, Gaussian Linear Uniform (10D), and SLCP. The reported evaluation metric is C2ST, where lower is better and $s(x,t):=\nabla_x \log p_t(x)$9 indicates perfect posterior estimation. The paper states that NPSE and NLSE are “often comparable or superior to state-of-the-art sequential methods” such as SNPE, SNLE, and SNRE, especially at small budgets around 1000 simulations (Sharrock et al., 2022).

In probabilistic time-series imputation, the conditioning variable is the observed subseries. CSDI defines the reverse process directly for the missing values conditioned on observed values, masks, timestamps, and feature embeddings, and uses 2D attention across temporal and feature axes. The paper reports that CSDI “improves by 40-65% over existing probabilistic imputation methods on popular performance metrics,” and that deterministic imputation “reduces the error by 5-20% compared to the state-of-the-art deterministic imputation methods.” For CRPS on healthcare data with $$s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),$$0 missingness, the reported values are $$s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),$$1 for CSDI, versus $$s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),$$2 for unconditional diffusion; for MAE the reported CSDI values are $$s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),$$3 (Tashiro et al., 2021).

In inverse physics, conditioning is imposed by fixing the terminal state and integrating backward through an approximate inverse simulator plus a learned score correction. “Solving Inverse Physics Problems with Score Matching” uses the update

$$s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),$$4

and shows that single-step training is equivalent to score matching, while multi-step rollout training relates to maximum-likelihood training of a corresponding probability flow. The reverse-time SDE version provides posterior sampling, whereas the probability-flow ODE yields a deterministic ML trajectory (Holzschuh et al., 2023).

5. Discrete and categorical analogues

For categorical data, the gradient $$s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),$$5 is not properly defined, so the conditional backward-in-time score is replaced by conditional marginals and reverse jump intensities. In score-based continuous-time discrete diffusion, the reverse-time CTMC has rate matrix

$$s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),$$6

and under single-coordinate jumps the learned reverse rate is

$$s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),$$7

The log-ratio

$$s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),$$8

plays the role of a conditional backward-in-time “score” analogue (Sun et al., 2022).

Learning proceeds by matching singleton conditional marginals. The paper states that matching $$s_{\mathrm{cond}}(x,y,t):=\nabla_x \log p_t(x\mid y),$$9 for all $$s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).$$0 and $$s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).$$1 is sufficient to match the joint distribution, and proposes the simplified objective

$$s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).$$2

An implicit parameterization through $$s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).$$3 then yields analytical reverse steps (Sun et al., 2022).

Empirically, the same framework reports strong results on both synthetic and real-world benchmarks. On CIFAR-10 in VQ code space, SDDM-VQ achieves IS $$s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).$$4 and FID $$s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).$$5, improving over D3PM-VQ with IS $$s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).$$6 and FID $$s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).$$7. At 50 reverse steps, the analytical sampler reports FID $$s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).$$8, whereas D3PM reports FID $$s(\theta,t\mid x):=\nabla_\theta \log p_t(\theta\mid x).$$9. On monophonic music, SDDM reports Hellinger Distance $$\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),$$0 and Proportion of Outliers $$\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),$$1 (Sun et al., 2022).

6. Exact conditional identities, variance reduction, and matrix gating

Several recent works replace approximate conditional-score surrogates with exact or variance-controlled identities. For a Gaussian mixture prior

$$\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),$$2

and linear-Gaussian observations $$\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),$$3, $$\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),$$4, “Exact Conditional Score-Guided Generative Modeling for Amortized Inference in Uncertainty Quantification” derives a closed-form conditional score

$$\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),$$5

as a mixture of componentwise scores and likelihood corrections weighted by exact responsibilities $$\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),$$6. The reverse-time probability-flow ODE

$$\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),$$7

is then used as a training-free conditional sampler. The same paper uses the resulting noise-labeled pairs $$\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),$$8 to train a feedforward network $$\nabla_\theta \log p(\theta\mid x)=\nabla_\theta \log p(x\mid \theta)+\nabla_\theta \log p(\theta),$$9 for amortized conditional inference. Reported numerical results include KL divergence “$p_t(\cdot\mid x)$0” in a 1D bimodal conditional example and projection KLs between $p_t(\cdot\mid x)$1 and $p_t(\cdot\mid x)$2 in a 20D two-mode GMM example (Zhang et al., 23 Jun 2025).

For affine diffusions, “Variance-Reduced Diffusion Sampling via Conditional Score Expectation Identity” proves the exact Conditional Score Expectation identity

$p_t(\cdot\mid x)$3

This expresses the time-$p_t(\cdot\mid x)$4 score as a conditional expectation of the initial score under the forward dynamics. The paper then constructs a Self-Normalized Importance Sampling estimator and blends it with a Tweedie estimator through a state–time dependent convex combination. Reported examples include a 24D Navier–Stokes posterior, where MMD to MALA is $p_t(\cdot\mid x)$5 for Tweedie and $p_t(\cdot\mid x)$6 for the blend, with KSD $p_t(\cdot\mid x)$7 versus $p_t(\cdot\mid x)$8; and an MNIST deblurring posterior in a 15D PCA space, where PSNR improves from $p_t(\cdot\mid x)$9 dB to $$\mathbb{E}_{p_t(x\mid y)}\!\left[\nabla_x \log p_t(x\mid y)\right]=0.$$0 dB and posterior coverage from $$\mathbb{E}_{p_t(x\mid y)}\!\left[\nabla_x \log p_t(x\mid y)\right]=0.$$1 to $$\mathbb{E}_{p_t(x\mid y)}\!\left[\nabla_x \log p_t(x\mid y)\right]=0.$$2 (Duston et al., 4 Jan 2026).

“Laplace–Fisher Gate Identities for Optimal Matrix-Gated Blended Score Estimation” further generalizes scalar blending to matrix-valued gates for OU diffusion reversal. With $$\mathbb{E}_{p_t(x\mid y)}\!\left[\nabla_x \log p_t(x\mid y)\right]=0.$$3, $$\mathbb{E}_{p_t(x\mid y)}\!\left[\nabla_x \log p_t(x\mid y)\right]=0.$$4, and $$\mathbb{E}_{p_t(x\mid y)}\!\left[\nabla_x \log p_t(x\mid y)\right]=0.$$5, the variance-optimal gate is

$$\mathbb{E}_{p_t(x\mid y)}\!\left[\nabla_x \log p_t(x\mid y)\right]=0.$$6

Because the Tweedie–TSI disagreement has conditional mean zero, the gate changes variance without changing expected value. The paper states that LFGI improves posterior-density calibration and sampling diagnostics relative to the other tested score-estimator classes, and that known-evidence experiments check absolute calibration in Gaussian and non-Gaussian settings (Duston et al., 23 Jun 2026).

A plausible implication is that the recent literature is moving from heuristic conditional guidance toward three increasingly stringent regimes: direct conditional score learning, exact conditional score construction under tractable model classes, and variance-optimized blending identities for reverse-time sampling. Across these regimes, the defining invariant remains the same: the conditional backward-in-time score is the object that makes reverse dynamics target a conditional law rather than an unconditional marginal.