Diffusion Contrastive Divergence in EBMs

• Diffusion Contrastive Divergence is defined as the contraction of the KL divergence via a predefined, parameter-free diffusion process applied to both data and model distributions.
• It generalizes classical Contrastive Divergence by substituting the model-dependent Langevin dynamics with a fixed diffusion, thereby avoiding complex MCMC sampling during training.
• Empirical evaluations show DCD's improved performance in synthetic density modeling, image denoising, and generation, despite requiring higher-order differentiability and memory-intensive computations.

Diffusion Contrastive Divergence (DCD) is a family of training objectives for energy-based models (EBMs) in which the model-dependent Langevin dynamics of classical Contrastive Divergence (CD) are replaced by a predefined, EBM-parameter-free diffusion process. In its canonical formulation, DCD measures the amount by which a shared diffusion contracts the Kullback–Leibler divergence between the data distribution and the model distribution, thereby reinterpreting CD as one member of a broader diffusion-induced divergence family rather than as a procedure tied specifically to short-run MCMC (Luo et al., 2023). The term has since coexisted with several adjacent but technically distinct lines of work—contrastive objectives for denoisers, joint EBM–diffusion training, and diffusion-based representation learning—so the literature surrounding “DCD” requires careful disambiguation (Yoon et al., 2023).

1. Classical contrastive divergence and the motivation for DCD

Classical CD trains an EBM by replacing the exact negative phase of maximum-likelihood learning with samples produced by a short Markov chain initialized at the data. For an EBM written as

$p_\theta(x) = \frac{\exp(f_\theta(x))}{Z_\theta},$

the maximum-likelihood gradient has the familiar positive-phase minus negative-phase form,

$$\frac{\partial}{\partial \theta}\mathbb{E}_{p_d}\log \frac{\exp(f_\theta(x))}{Z_\theta} = \mathbb{E}_{p_d}\frac{\partial}{\partial \theta} f_\theta(x) - \mathbb{E}_{p_\theta}\frac{\partial}{\partial \theta}f_\theta(x).$$

In standard CD, the model expectation is approximated by a short MCMC chain, typically Langevin dynamics,

$$d x_t = \frac{1}{2}\nabla_{x_t}\log p_\theta(x_t) d t + d \bm{w}_t,$$

and the objective is written as

$$\mathcal{D}_{CD}(p_d,p_\theta) = \mathcal{D}_{KL}(p_d, p_\theta) - \mathcal{D}_{KL}(p_{d,\theta}^{(T)}, p_\theta).$$

The central difficulty is that the short-run distribution $p_{d,\theta}^{(T)}$ depends on $\theta$, so differentiating CD introduces an additional term,

$$\frac{\partial}{\partial \theta}\mathcal{D}_{CD}(p_d,p_\theta) = \mathbb{E}_{ p_{d,\theta}^{(T)}}\bigg[\frac{\partial}{\partial \theta}f_\theta(x) \bigg] -\mathbb{E}_{ p_d}\bigg[\frac{\partial}{\partial \theta} f_\theta(x)\bigg] -\mathbb{E}_{p_{d,\theta}^{(T)}}\bigg[\log p_\theta(x) \frac{\partial}{\partial \theta}\log p_{d,\theta}^{(T)}(x) \bigg],$$

and that final term is non-negligible in general (Luo et al., 2023).

A complementary line of analysis showed that CD is not best understood as approximate descent on a fixed KL-difference objective. Instead, when the transition kernel is reversible and satisfies detailed balance with respect to $p_\theta$, the usual CD update is exactly proportional to the gradient of a binary cross-entropy objective that classifies whether a Markov trajectory is presented in forward order or time-reversed. That derivation also yields a correction weight for inexact chains such as finite-step Langevin dynamics (Yair et al., 2020). This perspective does not define DCD, but it clarifies why a reformulation of CD in diffusion-theoretic terms became attractive: the classical short-run MCMC negative phase is both computationally burdensome and theoretically awkward.

2. Formal definition of Diffusion Contrastive Divergence

DCD starts from a general diffusion process

$d x_t = \bm{F}(x_t,t)d t + \bm{G}(t)d \bm{w}_t,$

whose marginal density evolves by the Fokker–Planck equation

$$\frac{d}{d t}p(x,t) = - \langle \nabla_{x}, p(x,t)\bm{F}(x,t)\rangle + \frac{1}{2}\bm{G}^2(t)\Delta_{x} p(x,t), \qquad p(x,0)=p_0(x).$$

The decisive restriction is that the diffusion be EBM-parameter-free: $$\frac{\partial}{\partial \theta}\mathbb{E}_{p_d}\log \frac{\exp(f_\theta(x))}{Z_\theta} = \mathbb{E}_{p_d}\frac{\partial}{\partial \theta} f_\theta(x) - \mathbb{E}_{p_\theta}\frac{\partial}{\partial \theta}f_\theta(x).$$0 and $$\frac{\partial}{\partial \theta}\mathbb{E}_{p_d}\log \frac{\exp(f_\theta(x))}{Z_\theta} = \mathbb{E}_{p_d}\frac{\partial}{\partial \theta} f_\theta(x) - \mathbb{E}_{p_\theta}\frac{\partial}{\partial \theta}f_\theta(x).$$1 do not depend on $$\frac{\partial}{\partial \theta}\mathbb{E}_{p_d}\log \frac{\exp(f_\theta(x))}{Z_\theta} = \mathbb{E}_{p_d}\frac{\partial}{\partial \theta} f_\theta(x) - \mathbb{E}_{p_\theta}\frac{\partial}{\partial \theta}f_\theta(x).$$2. Under that condition, the diffused data distribution and the diffused model distribution are transported by the same known process, and the problematic model-dependent MCMC derivative of classical CD is avoided (Luo et al., 2023).

The core DCD divergence is

$$\frac{\partial}{\partial \theta}\mathbb{E}_{p_d}\log \frac{\exp(f_\theta(x))}{Z_\theta} = \mathbb{E}_{p_d}\frac{\partial}{\partial \theta} f_\theta(x) - \mathbb{E}_{p_\theta}\frac{\partial}{\partial \theta}f_\theta(x).$$3

where $$\frac{\partial}{\partial \theta}\mathbb{E}_{p_d}\log \frac{\exp(f_\theta(x))}{Z_\theta} = \mathbb{E}_{p_d}\frac{\partial}{\partial \theta} f_\theta(x) - \mathbb{E}_{p_\theta}\frac{\partial}{\partial \theta}f_\theta(x).$$4 and $$\frac{\partial}{\partial \theta}\mathbb{E}_{p_d}\log \frac{\exp(f_\theta(x))}{Z_\theta} = \mathbb{E}_{p_d}\frac{\partial}{\partial \theta} f_\theta(x) - \mathbb{E}_{p_\theta}\frac{\partial}{\partial \theta}f_\theta(x).$$5 are the time-$$\frac{\partial}{\partial \theta}\mathbb{E}_{p_d}\log \frac{\exp(f_\theta(x))}{Z_\theta} = \mathbb{E}_{p_d}\frac{\partial}{\partial \theta} f_\theta(x) - \mathbb{E}_{p_\theta}\frac{\partial}{\partial \theta}f_\theta(x).$$6 marginals obtained by diffusing the data distribution and model distribution under the same process. The main theorem gives the integral representation

$$\frac{\partial}{\partial \theta}\mathbb{E}_{p_d}\log \frac{\exp(f_\theta(x))}{Z_\theta} = \mathbb{E}_{p_d}\frac{\partial}{\partial \theta} f_\theta(x) - \mathbb{E}_{p_\theta}\frac{\partial}{\partial \theta}f_\theta(x).$$7

This immediately implies nonnegativity,

$$\frac{\partial}{\partial \theta}\mathbb{E}_{p_d}\log \frac{\exp(f_\theta(x))}{Z_\theta} = \mathbb{E}_{p_d}\frac{\partial}{\partial \theta} f_\theta(x) - \mathbb{E}_{p_\theta}\frac{\partial}{\partial \theta}f_\theta(x).$$8

Under the assumptions stated in the paper, DCD is therefore a proper probability divergence, and it can be interpreted as the accumulated discrepancy between diffused scores along the diffusion path (Luo et al., 2023).

Within this framework, classical CD appears as a special case. If the diffusion is chosen to be the EBM-induced Langevin dynamics,

$$\frac{\partial}{\partial \theta}\mathbb{E}_{p_d}\log \frac{\exp(f_\theta(x))}{Z_\theta} = \mathbb{E}_{p_d}\frac{\partial}{\partial \theta} f_\theta(x) - \mathbb{E}_{p_\theta}\frac{\partial}{\partial \theta}f_\theta(x).$$9

then $$d x_t = \frac{1}{2}\nabla_{x_t}\log p_\theta(x_t) d t + d \bm{w}_t,$$0 is stationary, so $$d x_t = \frac{1}{2}\nabla_{x_t}\log p_\theta(x_t) d t + d \bm{w}_t,$$1, and DCD reduces to CD. This is the sense in which DCD generalizes CD rather than merely replacing one sampler with another (Luo et al., 2023).

3. Practical instantiation: variance-exploding diffusion and the one-step objective

The practical version developed in the original DCD paper is variance-exploding diffusion,

$$d x_t = \frac{1}{2}\nabla_{x_t}\log p_\theta(x_t) d t + d \bm{w}_t,$$2

This process has no drift and simply adds Gaussian noise. The paper writes the transition kernel as

$$d x_t = \frac{1}{2}\nabla_{x_t}\log p_\theta(x_t) d t + d \bm{w}_t,$$3

Because the transition is explicit and parameter-free, drawing $$d x_t = \frac{1}{2}\nabla_{x_t}\log p_\theta(x_t) d t + d \bm{w}_t,$$4 from data is inexpensive: sample $$d x_t = \frac{1}{2}\nabla_{x_t}\log p_\theta(x_t) d t + d \bm{w}_t,$$5 and perturb it with Gaussian noise (Luo et al., 2023).

For parameter-free diffusions, DCD minimization is equivalent to minimizing

$$d x_t = \frac{1}{2}\nabla_{x_t}\log p_\theta(x_t) d t + d \bm{w}_t,$$6

The difficulty shifts from MCMC sampling to the evaluation of the evolved model energy $$d x_t = \frac{1}{2}\nabla_{x_t}\log p_\theta(x_t) d t + d \bm{w}_t,$$7. In general that evolution is governed by a PDE. For VE diffusion, the paper derives

$$d x_t = \frac{1}{2}\nabla_{x_t}\log p_\theta(x_t) d t + d \bm{w}_t,$$8

Rather than solving this long-time evolution exactly, the practical algorithm uses a small-$$d x_t = \frac{1}{2}\nabla_{x_t}\log p_\theta(x_t) d t + d \bm{w}_t,$$9, one-step approximation, yielding

$$\mathcal{D}_{CD}(p_d,p_\theta) = \mathcal{D}_{KL}(p_d, p_\theta) - \mathcal{D}_{KL}(p_{d,\theta}^{(T)}, p_\theta).$$0

The first bracketed term has the same structure as score matching, while the second term is a contrastive energy difference between clean and noised data. The paper explicitly connects DCD to Diffusion Recovery Likelihood, identifying DRL as a special case of the same KL-contraction form under Gaussian perturbation (Luo et al., 2023).

A time-dependent variant is also described, in which one trains a time-dependent energy model $$\mathcal{D}_{CD}(p_d,p_\theta) = \mathcal{D}_{KL}(p_d, p_\theta) - \mathcal{D}_{KL}(p_{d,\theta}^{(T)}, p_\theta).$$1 over diffused data distributions by sampling $$\mathcal{D}_{CD}(p_d,p_\theta) = \mathcal{D}_{KL}(p_d, p_\theta) - \mathcal{D}_{KL}(p_{d,\theta}^{(T)}, p_\theta).$$2, diffusing data to $$\mathcal{D}_{CD}(p_d,p_\theta) = \mathcal{D}_{KL}(p_d, p_\theta) - \mathcal{D}_{KL}(p_{d,\theta}^{(T)}, p_\theta).$$3, computing the relevant DCD objective, and updating $$\mathcal{D}_{CD}(p_d,p_\theta) = \mathcal{D}_{KL}(p_d, p_\theta) - \mathcal{D}_{KL}(p_{d,\theta}^{(T)}, p_\theta).$$4 by gradient descent (Luo et al., 2023).

4. Computational characteristics, implementation, and algorithmic trade-offs

DCD changes the computational bottleneck of EBM training. Classical CD requires sequential, model-dependent MCMC, and its quality depends on chain length and mixing. DCD-VE eliminates EBM-driven MCMC and replaces it with direct Gaussian perturbation, but it requires higher-order differentiation through the energy network. In particular, the practical loss depends on $$\mathcal{D}_{CD}(p_d,p_\theta) = \mathcal{D}_{KL}(p_d, p_\theta) - \mathcal{D}_{KL}(p_{d,\theta}^{(T)}, p_\theta).$$5 and the Laplacian $$\mathcal{D}_{CD}(p_d,p_\theta) = \mathcal{D}_{KL}(p_d, p_\theta) - \mathcal{D}_{KL}(p_{d,\theta}^{(T)}, p_\theta).$$6 (Luo et al., 2023).

In low dimensions, the Laplacian can be computed directly. In high dimensions, the paper uses Hutchinson trace estimation,

$$\mathcal{D}_{CD}(p_d,p_\theta) = \mathcal{D}_{KL}(p_d, p_\theta) - \mathcal{D}_{KL}(p_{d,\theta}^{(T)}, p_\theta).$$7

where $$\mathcal{D}_{CD}(p_d,p_\theta) = \mathcal{D}_{KL}(p_d, p_\theta) - \mathcal{D}_{KL}(p_{d,\theta}^{(T)}, p_\theta).$$8. The resulting trade-off is explicit in the paper: DCD is often faster in wall-clock time than CD, but it is more memory-intensive because of second-order derivative computation (Luo et al., 2023).

The original experiments instantiate this trade-off in three regimes. In 2D synthetic density modeling, DCD-VE uses a 4-layer MLP with 300 hidden units per layer, GELU activations, $$\mathcal{D}_{CD}(p_d,p_\theta) = \mathcal{D}_{KL}(p_d, p_\theta) - \mathcal{D}_{KL}(p_{d,\theta}^{(T)}, p_\theta).$$9, $p_{d,\theta}^{(T)}$0, Adam with learning rate $p_{d,\theta}^{(T)}$1, batch size $p_{d,\theta}^{(T)}$2, and $p_{d,\theta}^{(T)}$3 iterations. The CD baseline uses Langevin step size $p_{d,\theta}^{(T)}$4 and $p_{d,\theta}^{(T)}$5 Langevin steps; the PCD baseline uses replay buffer size $p_{d,\theta}^{(T)}$6, step size $p_{d,\theta}^{(T)}$7, $p_{d,\theta}^{(T)}$8 MCMC steps, and replay update frequency $p_{d,\theta}^{(T)}$9 (Luo et al., 2023).

For image denoising, the paper uses Wide ResNet architectures with SiLU and no normalization. MNIST and FashionMNIST use depth $\theta$0 and widen factor $\theta$1; CIFAR10 and SVHN use depth $\theta$2 and widen factor $\theta$3. Inputs are scaled to $\theta$4, training data are additionally corrupted with Gaussian noise $\theta$5, and DCD-VE uses $\theta$6, $\theta$7, Adam, learning rate $\theta$8, and $\theta$9 (Luo et al., 2023).

For image generation on CelebA $$\frac{\partial}{\partial \theta}\mathcal{D}_{CD}(p_d,p_\theta) = \mathbb{E}_{ p_{d,\theta}^{(T)}}\bigg[\frac{\partial}{\partial \theta}f_\theta(x) \bigg] -\mathbb{E}_{ p_d}\bigg[\frac{\partial}{\partial \theta} f_\theta(x)\bigg] -\mathbb{E}_{p_{d,\theta}^{(T)}}\bigg[\log p_\theta(x) \frac{\partial}{\partial \theta}\log p_{d,\theta}^{(T)}(x) \bigg],$$0, the paper trains a time-dependent EBM with a residual/UNet-style architecture from an EDM VP backbone plus an extra SiLU before the last pooling layer. The forward process uses VE diffusion with $$\frac{\partial}{\partial \theta}\mathcal{D}_{CD}(p_d,p_\theta) = \mathbb{E}_{ p_{d,\theta}^{(T)}}\bigg[\frac{\partial}{\partial \theta}f_\theta(x) \bigg] -\mathbb{E}_{ p_d}\bigg[\frac{\partial}{\partial \theta} f_\theta(x)\bigg] -\mathbb{E}_{p_{d,\theta}^{(T)}}\bigg[\log p_\theta(x) \frac{\partial}{\partial \theta}\log p_{d,\theta}^{(T)}(x) \bigg],$$1, training samples time as $$\frac{\partial}{\partial \theta}\mathcal{D}_{CD}(p_d,p_\theta) = \mathbb{E}_{ p_{d,\theta}^{(T)}}\bigg[\frac{\partial}{\partial \theta}f_\theta(x) \bigg] -\mathbb{E}_{ p_d}\bigg[\frac{\partial}{\partial \theta} f_\theta(x)\bigg] -\mathbb{E}_{p_{d,\theta}^{(T)}}\bigg[\log p_\theta(x) \frac{\partial}{\partial \theta}\log p_{d,\theta}^{(T)}(x) \bigg],$$2, and reverse sampling uses a Heun solver with 18 discretized noise levels from $$\frac{\partial}{\partial \theta}\mathcal{D}_{CD}(p_d,p_\theta) = \mathbb{E}_{ p_{d,\theta}^{(T)}}\bigg[\frac{\partial}{\partial \theta}f_\theta(x) \bigg] -\mathbb{E}_{ p_d}\bigg[\frac{\partial}{\partial \theta} f_\theta(x)\bigg] -\mathbb{E}_{p_{d,\theta}^{(T)}}\bigg[\log p_\theta(x) \frac{\partial}{\partial \theta}\log p_{d,\theta}^{(T)}(x) \bigg],$$3 to $$\frac{\partial}{\partial \theta}\mathcal{D}_{CD}(p_d,p_\theta) = \mathbb{E}_{ p_{d,\theta}^{(T)}}\bigg[\frac{\partial}{\partial \theta}f_\theta(x) \bigg] -\mathbb{E}_{ p_d}\bigg[\frac{\partial}{\partial \theta} f_\theta(x)\bigg] -\mathbb{E}_{p_{d,\theta}^{(T)}}\bigg[\log p_\theta(x) \frac{\partial}{\partial \theta}\log p_{d,\theta}^{(T)}(x) \bigg],$$4 (Luo et al., 2023).

5. Empirical results, performance profile, and limitations

The empirical results reported for DCD divide naturally into synthetic density learning, denoising, and image generation. On seven 2D datasets—Swissroll, Circles, Rings, Moons, 8 Gaussians, 2 Spirals, and Checkerboard—the evaluation metric is the score matching loss

$$\frac{\partial}{\partial \theta}\mathcal{D}_{CD}(p_d,p_\theta) = \mathbb{E}_{ p_{d,\theta}^{(T)}}\bigg[\frac{\partial}{\partial \theta}f_\theta(x) \bigg] -\mathbb{E}_{ p_d}\bigg[\frac{\partial}{\partial \theta} f_\theta(x)\bigg] -\mathbb{E}_{p_{d,\theta}^{(T)}}\bigg[\log p_\theta(x) \frac{\partial}{\partial \theta}\log p_{d,\theta}^{(T)}(x) \bigg],$$5

DCD-VE outperforms CD and PCD on all seven datasets. The reported values include $$\frac{\partial}{\partial \theta}\mathcal{D}_{CD}(p_d,p_\theta) = \mathbb{E}_{ p_{d,\theta}^{(T)}}\bigg[\frac{\partial}{\partial \theta}f_\theta(x) \bigg] -\mathbb{E}_{ p_d}\bigg[\frac{\partial}{\partial \theta} f_\theta(x)\bigg] -\mathbb{E}_{p_{d,\theta}^{(T)}}\bigg[\log p_\theta(x) \frac{\partial}{\partial \theta}\log p_{d,\theta}^{(T)}(x) \bigg],$$6 on Swissroll for DCD-VE versus $$\frac{\partial}{\partial \theta}\mathcal{D}_{CD}(p_d,p_\theta) = \mathbb{E}_{ p_{d,\theta}^{(T)}}\bigg[\frac{\partial}{\partial \theta}f_\theta(x) \bigg] -\mathbb{E}_{ p_d}\bigg[\frac{\partial}{\partial \theta} f_\theta(x)\bigg] -\mathbb{E}_{p_{d,\theta}^{(T)}}\bigg[\log p_\theta(x) \frac{\partial}{\partial \theta}\log p_{d,\theta}^{(T)}(x) \bigg],$$7 for both CD and PCD, $$\frac{\partial}{\partial \theta}\mathcal{D}_{CD}(p_d,p_\theta) = \mathbb{E}_{ p_{d,\theta}^{(T)}}\bigg[\frac{\partial}{\partial \theta}f_\theta(x) \bigg] -\mathbb{E}_{ p_d}\bigg[\frac{\partial}{\partial \theta} f_\theta(x)\bigg] -\mathbb{E}_{p_{d,\theta}^{(T)}}\bigg[\log p_\theta(x) \frac{\partial}{\partial \theta}\log p_{d,\theta}^{(T)}(x) \bigg],$$8 on Rings versus $$\frac{\partial}{\partial \theta}\mathcal{D}_{CD}(p_d,p_\theta) = \mathbb{E}_{ p_{d,\theta}^{(T)}}\bigg[\frac{\partial}{\partial \theta}f_\theta(x) \bigg] -\mathbb{E}_{ p_d}\bigg[\frac{\partial}{\partial \theta} f_\theta(x)\bigg] -\mathbb{E}_{p_{d,\theta}^{(T)}}\bigg[\log p_\theta(x) \frac{\partial}{\partial \theta}\log p_{d,\theta}^{(T)}(x) \bigg],$$9 for both baselines, and $p_\theta$0 on Checkerboard versus $p_\theta$1 for CD and $p_\theta$2 for PCD (Luo et al., 2023).

In image denoising, the metric is average RMSE after adding Gaussian noise at levels $p_\theta$3, $p_\theta$4, and $p_\theta$5. DCD consistently outperforms CD across MNIST, FashionMNIST, CIFAR10, and SVHN. The gap becomes especially large at high noise: on CIFAR10, DCD reports $p_\theta$6 at high noise versus $p_\theta$7 for CD; on SVHN, the corresponding values are $p_\theta$8 versus $p_\theta$9 (Luo et al., 2023). This suggests that the learned energy landscape remains useful farther from the clean data manifold, although that inference concerns interpretation rather than a theorem.

In image generation on CelebA $d x_t = \bm{F}(x_t,t)d t + \bm{G}(t)d \bm{w}_t,$0, EBM-DCD reports FID $d x_t = \bm{F}(x_t,t)d t + \bm{G}(t)d \bm{w}_t,$1. The paper places that result as better than ABP at $d x_t = \bm{F}(x_t,t)d t + \bm{G}(t)d \bm{w}_t,$2, ABP-SRI at $d x_t = \bm{F}(x_t,t)d t + \bm{G}(t)d \bm{w}_t,$3, VAE at $d x_t = \bm{F}(x_t,t)d t + \bm{G}(t)d \bm{w}_t,$4, and Glow at $d x_t = \bm{F}(x_t,t)d t + \bm{G}(t)d \bm{w}_t,$5, comparable to DCGAN at $d x_t = \bm{F}(x_t,t)d t + \bm{G}(t)d \bm{w}_t,$6 and EBM-FCE at $d x_t = \bm{F}(x_t,t)d t + \bm{G}(t)d \bm{w}_t,$7, and worse than GEBM at $d x_t = \bm{F}(x_t,t)d t + \bm{G}(t)d \bm{w}_t,$8 and CoopFlow(T=30) at $d x_t = \bm{F}(x_t,t)d t + \bm{G}(t)d \bm{w}_t,$9 (Luo et al., 2023). The paper therefore presents DCD generation as viable rather than leading.

Two limitations are explicit. First, DCD requires higher-order derivatives, so the energy model must be at least twice differentiable. Second, exact long-time energy evolution is difficult, and the practical method relies on a one-step approximation rather than a full consistency result for long diffusion times (Luo et al., 2023). A plausible implication is that DCD is especially attractive when MCMC instability dominates and second-order autodiff is feasible.

6. Related formulations, neighboring methods, and common confusions

The phrase “Diffusion Contrastive Divergence” now sits beside several technically different frameworks. The following distinctions are essential.

Method	Core mechanism	Relation to DCD
DCD (Luo et al., 2023)	KL contraction under parameter-free diffusion for EBM training	Canonical use of the term
GCD (Yoon et al., 2023)	Joint minimax training of an EBM and a trainable sampler, including diffusion samplers	Generalizes CD by replacing MCMC with a learned sampler
CDL (Wu et al., 2024)	Logistic classification between clean and more heavily noised marginals	Adjacent contrastive diffusion training, not classical CD
DCR (Han et al., 5 Mar 2026)	InfoNCE-style loss on predicted noises in diffusion reconstruction	Contrastive regularization, not contrastive divergence
DCG (Zhang et al., 12 Mar 2025)	DDPM-style latent diffusion plus contrastive alignment for incomplete multi-view clustering	Diffusion with contrastive losses, not CD for EBMs
DCDM (Zhu et al., 15 May 2026)	Dynamic semantic chunking for discrete diffusion language modeling	Acronym overlap only

Generalized Contrastive Divergence (GCD) is the nearest extension in spirit. It replaces the MCMC negative phase with an arbitrary trainable sampler $$\frac{d}{d t}p(x,t) = - \langle \nabla_{x}, p(x,t)\bm{F}(x,t)\rangle + \frac{1}{2}\bm{G}^2(t)\Delta_{x} p(x,t), \qquad p(x,0)=p_0(x).$$0, and when $$\frac{d}{d t}p(x,t) = - \langle \nabla_{x}, p(x,t)\bm{F}(x,t)\rangle + \frac{1}{2}\bm{G}^2(t)\Delta_{x} p(x,t), \qquad p(x,0)=p_0(x).$$1 is instantiated as a diffusion model the joint training becomes

$$\frac{d}{d t}p(x,t) = - \langle \nabla_{x}, p(x,t)\bm{F}(x,t)\rangle + \frac{1}{2}\bm{G}^2(t)\Delta_{x} p(x,t), \qquad p(x,0)=p_0(x).$$2

The paper formulates this joint learning as a minimax problem and interprets it through inverse reinforcement learning, where the energy is a negative reward and the diffusion model is a policy (Yoon et al., 2023). Relative to DCD, the difference is structural: DCD uses a predefined diffusion and optimizes a divergence on the EBM, whereas GCD learns the sampler itself.

A second adjacent line reinterprets diffusion denoisers as implicit classifiers. “Your Diffusion Model is Secretly a Noise Classifier and Benefits from Contrastive Training” introduces Contrastive Diffusion Loss (CDL), a logistic objective

$$\frac{d}{d t}p(x,t) = - \langle \nabla_{x}, p(x,t)\bm{F}(x,t)\rangle + \frac{1}{2}\bm{G}^2(t)\Delta_{x} p(x,t), \qquad p(x,0)=p_0(x).$$3

which discriminates between the clean data distribution $$\frac{d}{d t}p(x,t) = - \langle \nabla_{x}, p(x,t)\bm{F}(x,t)\rangle + \frac{1}{2}\bm{G}^2(t)\Delta_{x} p(x,t), \qquad p(x,0)=p_0(x).$$4 and a noisier marginal $$\frac{d}{d t}p(x,t) = - \langle \nabla_{x}, p(x,t)\bm{F}(x,t)\rangle + \frac{1}{2}\bm{G}^2(t)\Delta_{x} p(x,t), \qquad p(x,0)=p_0(x).$$5. The paper frames CDL as a self-supervised binary classification or density-ratio objective between two points on the diffusion path, and reports improvements in OOD denoising and especially in parallel sampling, but it does not define an energy-based diffusion model or a contrastive-divergence objective in the classical sense (Wu et al., 2024).

Other recent works use “contrastive” in representation-learning rather than EBM-training senses. DCR places an InfoNCE-style objective directly in diffusion predicted-noise space to enhance CLIP representations, explicitly arguing that a naive weighted sum of contrastive and reconstruction losses suffers from gradient conflict, with $$\frac{d}{d t}p(x,t) = - \langle \nabla_{x}, p(x,t)\bm{F}(x,t)\rangle + \frac{1}{2}\bm{G}^2(t)\Delta_{x} p(x,t), \qquad p(x,0)=p_0(x).$$6 of training steps having negative gradient cosine similarity (Han et al., 5 Mar 2026). DCG combines DDPM-style latent diffusion, generated-view contrastive loss, mutual-information alignment, and KL-based clustering objectives for incomplete multi-view clustering, but it contains no energy function, no partition function, and no CD-style positive-versus-negative phase (Zhang et al., 12 Mar 2025).

A recurrent source of confusion is acronymic rather than conceptual. “Dynamic Chunking for Diffusion LLMs” introduces DCDM, the Dynamic Chunking Diffusion Model, for discrete diffusion language modeling. That work explicitly states that it is not about “Diffusion Contrastive Divergence (DCD)” in the usual sense; its contribution is dynamic semantic chunking and chunk-causal attention in block diffusion LLMs, not a contrastive-divergence training objective (Zhu et al., 15 May 2026).

Taken together, the literature supports a narrow and a broad usage. In the narrow sense, Diffusion Contrastive Divergence denotes the KL-contraction family for EBM training introduced in (Luo et al., 2023). In the broader sense, it has become a reference point for several diffusion methods that use contrastive, classifier-based, or adversarial signals. Maintaining that distinction is necessary for precise reading of current arXiv work.