Authentic Discrete Diffusion (ADD) Model

• Authentic Discrete Diffusion (ADD) is a diffusion framework that operates directly on one-hot encoded data, maintaining true diffusion dynamics in discrete settings.
• It employs a timestep-conditioned cross-entropy loss to bridge discriminative and generative learning, achieving significant performance gains in tasks like image classification.
• The model utilizes a rigorous CTMC formulation and an argmax-to-one-hot projection method to ensure precise likelihood matching and faithful reverse denoising.

Authentic Discrete Diffusion (ADD) is a diffusion framework designed to work directly on discrete data such as class labels and text tokens, while preserving the core structure of continuous denoising diffusion probabilistic models; in a broader information-theoretic and CTMC usage, an ADD model is a discrete diffusion model whose training losses and inference procedures are exactly tied to the data log-likelihood, mutual information, and the true reverse continuous-time Markov chain, rather than to heuristic masking rules or continuous relaxations (Li et al., 1 Oct 2025, Jeon et al., 28 Oct 2025, Noguerales et al., 6 Jul 2026).

1. Definition, motivation, and scope

Standard diffusion models operate on continuous vectors with Gaussian noise and MSE losses. Discrete data live on a categorical simplex with non-Euclidean geometry and mutually exclusive outcomes, making naive extensions of DDPM problematic. The ADD formulation in "Authentic Discrete Diffusion Model" (Li et al., 1 Oct 2025) is explicitly positioned against two families that the paper describes as inadequate for this setting: embedding-based diffusion, which maps discrete tokens to continuous embeddings and runs Gaussian diffusion there, and masked or BERT-style “discrete diffusion,” which the paper characterizes as pseudo discrete diffusion because it lacks a well-defined Gaussian Markov forward process and is not a true diffusion process in the sense of Sohl-Dickstein et al.

In that formulation, ADD is introduced to be “authentic” in two senses. First, it preserves the defining characteristics of diffusion models: a Gaussian forward noising process with a variance schedule and an iterative reverse denoising process. Second, it operates directly on float-encoded one-hot categorical vectors, not on continuous embeddings or masked tokens, and uses explicit discretization, argmax → one-hot, at each step (Li et al., 1 Oct 2025).

A broader theoretical usage appears in "Information-Theoretic Discrete Diffusion" (Jeon et al., 28 Oct 2025), which states that an “Authentic Discrete Diffusion (ADD)” model would be a discrete diffusion model whose training losses and inference procedures are not heuristic, but are exactly tied to the data log-likelihood and mutual information. "What Does a Discrete Diffusion Model Learn?" sharpens this criterion further: an authentic method is mathematically aligned with the true reverse CTMC induced by a chosen forward noising process, with a model output that is unambiguously interpreted as reverse jump rates or an equivalent coordinate such as denoiser, cavity, or score (Noguerales et al., 6 Jul 2026). This suggests a two-level description: ADD is both a concrete one-hot-space diffusion architecture and a criterion of mathematical faithfulness for discrete diffusion more generally.

2. One-hot-space diffusion model

For a $K$-class categorical variable, ADD represents a label as a one-hot vector

$$\mathbf{y}_0 \in \{0,1\}^K,\quad \sum_{k=1}^K y_0^{(k)} = 1.$$

For text, a sentence of length $N$ is represented as

$$\mathbf{Y}_0 = [\mathbf{y}_{0,1}, \dots, \mathbf{y}_{0,N}],\quad \mathbf{y}_{0,i}\in \{0,1\}^K.$$

These one-hot vectors are treated as real-valued vectors in $\mathbb{R}^K$ so that Gaussian noise can be added (Li et al., 1 Oct 2025).

The forward process uses the same functional form as DDPM, but applied directly to one-hot vectors: $$q(\mathbf{y}_t \mid \mathbf{y}_0) = \mathcal{N}\!\big(\mathbf{y}_t;\, \sqrt{\bar{\alpha}_t}\,\mathbf{y}_0,\,(1-\bar{\alpha}_t)\mathbf{I}\big).$$ For text, the same corruption is applied independently to each token: $$q(\mathbf{y}_{t,i} \mid \mathbf{y}_{0,i}) = \mathcal{N}\!\big(\mathbf{y}_{t,i}; \sqrt{\bar{\alpha}_t}\,\mathbf{y}_{0,i}, (1-\bar{\alpha}_t)\mathbf{I}\big),\quad i=1,\dots,N.$$ The reverse process during training predicts a categorical distribution over the clean one-hot label: $$p_\theta(\mathbf{y}_0 \mid \mathbf{y}_t, c) = \operatorname{Softmax}\big(f_\theta(\mathbf{y}_t, t, c)\big).$$ At inference, ADD enforces discrete structure via an argmax→one-hot projection at every step: $$\hat{\mathbf{y}}_0 = \operatorname{onehot}\!\Big(\arg\max_k\, p_\theta(y_0^{(k)} \mid \mathbf{y}_t, c)\Big),$$ followed by re-noising,

$$\mathbf{y}_{t-1} \sim \mathcal{N}\!\big(\sqrt{\alpha_{t-1}}\,\hat{\mathbf{y}}_0,\,(1-\alpha_{t-1})\mathbf{I}\big).$$

Iterating this loop from a large $$\mathbf{y}_0 \in \{0,1\}^K,\quad \sum_{k=1}^K y_0^{(k)} = 1.$$0 down to $$\mathbf{y}_0 \in \{0,1\}^K,\quad \sum_{k=1}^K y_0^{(k)} = 1.$$1 yields progressively sharper categorical predictions. The paper states that this combination yields an authentic diffusion in one-hot space and forms a direct bridge between discriminative tasks and generative tasks (Li et al., 1 Oct 2025).

3. Timestep-conditioned cross-entropy and the discriminative–generative bridge

A central novelty of ADD is the timestep-conditioned cross-entropy objective, which replaces the usual MSE noise regression of DDPMs: $$\mathbf{y}_0 \in \{0,1\}^K,\quad \sum_{k=1}^K y_0^{(k)} = 1.$$2 For multi-token text, the paper gives

$$\mathbf{y}_0 \in \{0,1\}^K,\quad \sum_{k=1}^K y_0^{(k)} = 1.$$3

Uniform sampling over timesteps trains the model to handle all noise levels, while the coefficient $$\mathbf{y}_0 \in \{0,1\}^K,\quad \sum_{k=1}^K y_0^{(k)} = 1.$$4 is motivated as a way to prevent the diffusion network from over-relying on the conditioning features $$\mathbf{y}_0 \in \{0,1\}^K,\quad \sum_{k=1}^K y_0^{(k)} = 1.$$5 (Li et al., 1 Oct 2025).

The paper treats this loss as the mechanism by which denoising becomes classification. In the discriminative view, the model predicts $$\mathbf{y}_0 \in \{0,1\}^K,\quad \sum_{k=1}^K y_0^{(k)} = 1.$$6 given a noisy label $$\mathbf{y}_0 \in \{0,1\}^K,\quad \sum_{k=1}^K y_0^{(k)} = 1.$$7 and features $$\mathbf{y}_0 \in \{0,1\}^K,\quad \sum_{k=1}^K y_0^{(k)} = 1.$$8. In the generative view, the same predictor is applied repeatedly through the diffusion chain. This is the sense in which the method “establishes a bridge between discriminative and generative learning” (Li et al., 1 Oct 2025).

The ablations reported in the paper are unusually sharp. An ADD variant with regressive, MSE-like loss for noise prediction reaches Top-1 $$\mathbf{y}_0 \in \{0,1\}^K,\quad \sum_{k=1}^K y_0^{(k)} = 1.$$9, whereas ADD with timestep-conditioned cross-entropy reaches Top-1 $N$0 at 400 epochs. The paper also reports that classifier-free guidance improves Top-1 from $N$1 to $N$2, and that argmax followed by one-hot projection improves Top-1 from $N$3 to $N$4. These results are used to support the claim that discrete cross-entropy on one-hot labels is essential, and that timestep weighting, classifier-free guidance, and argmax-plus-one-hot discretization all contribute measurable gains while maintaining diffusion behavior (Li et al., 1 Oct 2025).

4. Information-theoretic likelihood interpretation

"Information-Theoretic Discrete Diffusion" (Jeon et al., 28 Oct 2025) supplies a different notion of authenticity. Its starting point is a continuous-time Markov chain forward diffusion

$N$5

with reverse rate matrix

$N$6

The central object is the marginal ratio

$N$7

which plays the role analogous to the score $N$8 in continuous diffusion (Jeon et al., 28 Oct 2025).

For general CTMC diffusion, the paper introduces the Denoising Score Entropy loss and proves that its optimum is the true ratio $N$9. It then establishes the Information-Minimum Denoising Score Entropy relation: $$\mathbf{Y}_0 = [\mathbf{y}_{0,1}, \dots, \mathbf{y}_{0,N}],\quad \mathbf{y}_{0,i}\in \{0,1\}^K.$$0 together with an exact negative log-likelihood decomposition,

$$\mathbf{Y}_0 = [\mathbf{y}_{0,1}, \dots, \mathbf{y}_{0,N}],\quad \mathbf{y}_{0,i}\in \{0,1\}^K.$$1

in the full-noise limit under ergodicity. The paper’s explicit conclusion is that integrated DSE is equal to the true NLL in the score-optimal case; DSE is not merely a variational upper bound, but a tight and principled estimator of log-likelihood (Jeon et al., 28 Oct 2025).

For masked or absorbing diffusion, the paper gives a parallel theory based on Denoising Cross-Entropy. With

$$\mathbf{Y}_0 = [\mathbf{y}_{0,1}, \dots, \mathbf{y}_{0,N}],\quad \mathbf{y}_{0,i}\in \{0,1\}^K.$$2

it proves both DSE–DCE equivalence for absorbing processes and an Information-Minimum Denoising Cross-Entropy identity: $$\mathbf{Y}_0 = [\mathbf{y}_{0,1}, \dots, \mathbf{y}_{0,N}],\quad \mathbf{y}_{0,i}\in \{0,1\}^K.$$3 The corresponding NLL identity is

$$\mathbf{Y}_0 = [\mathbf{y}_{0,1}, \dots, \mathbf{y}_{0,N}],\quad \mathbf{y}_{0,i}\in \{0,1\}^K.$$4

The paper states this explicitly: DCE is an exact estimator of NLL for absorbing diffusion, not a mere variational bound. It further derives time-free likelihood formulas, conditional likelihood estimators for prompt–response tasks, and coupled Monte Carlo estimators of likelihood ratios (Jeon et al., 28 Oct 2025).

Within this information-theoretic vocabulary, ADD denotes a discrete diffusion model whose losses are distribution–loss matched in the same sense that Gaussian noise pairs with MSE and I-MMSE. This use of the term does not depend on one-hot Gaussian diffusion specifically; it depends on exact likelihood matching.

5. CTMC alignment, ELBO exactness, and coordinate semantics

A more general state-space account appears in "Foundations of Diffusion Models in General State Spaces" (Pauline et al., 4 Dec 2025). There, a genuine discrete diffusion is described as a Markov chain or CTMC on a finite state space, with forward one-step kernel

$$\mathbf{Y}_0 = [\mathbf{y}_{0,1}, \dots, \mathbf{y}_{0,N}],\quad \mathbf{y}_{0,i}\in \{0,1\}^K.$$5

continuous-time rate matrix

$$\mathbf{Y}_0 = [\mathbf{y}_{0,1}, \dots, \mathbf{y}_{0,N}],\quad \mathbf{y}_{0,i}\in \{0,1\}^K.$$6

and a reverse process derived from time reversal rather than introduced ad hoc. For masking or absorbing diffusion, the ELBO reduces to a weighted masked-language-modeling cross-entropy, which the paper presents as an ELBO-based maximum likelihood method rather than a heuristic masking objective (Pauline et al., 4 Dec 2025).

"What Does a Discrete Diffusion Model Learn?" develops the strongest formal criterion of authenticity (Noguerales et al., 6 Jul 2026). It derives the CTMC ELBO with boundary terms and proves the Oracle Distance theorem: the negative ELBO is exactly equal to the data entropy plus the path KL from the oracle reverse process to the learned one. It also proves that the unique optimizer is the conditional expectation of the true reverse jump rate given the current noisy state, and that its irreducible cost is

$$\mathbf{Y}_0 = [\mathbf{y}_{0,1}, \dots, \mathbf{y}_{0,N}],\quad \mathbf{y}_{0,i}\in \{0,1\}^K.$$7

In this framework, denoiser, cavity, and score are “one object in different coordinates,” and reading a neural network in the wrong coordinate changes the process being trained and sampled (Noguerales et al., 6 Jul 2026).

For token-factorizing noise, the paper gives three exact coordinates for the optimizer: denoiser, cavity, and score, with closed-form conversions among them. It also proves that denoiser and cavity coincide for masked diffusion but not for uniform diffusion, and that a denoiser parameterization makes the uniform ELBO diverge at initialization while the bridge plug-in stays finite. A plausible implication is that authenticity, in the strict CTMC sense, requires not only a legitimate forward process and an ELBO, but also a precise semantic contract for what the network outputs represent during both training and sampling (Noguerales et al., 6 Jul 2026).

6. Architectures, empirical results, applications, and nomenclature

In its direct architectural instantiation, ADD is implemented as a hybrid of a Transformer feature extractor for conditioning and a diffusion module that takes noisy one-hot vectors plus conditioning and outputs logits over classes or tokens. The full model reported in "Authentic Discrete Diffusion Model" has about $$\mathbf{Y}_0 = [\mathbf{y}_{0,1}, \dots, \mathbf{y}_{0,N}],\quad \mathbf{y}_{0,i}\in \{0,1\}^K.$$8M parameters, with about $$\mathbf{Y}_0 = [\mathbf{y}_{0,1}, \dots, \mathbf{y}_{0,N}],\quad \mathbf{y}_{0,i}\in \{0,1\}^K.$$9M in the Transformer encoder for images and about $\mathbb{R}^K$0M in the diffusion module. Training uses AdamW with learning rate $\mathbb{R}^K$1, weight decay $\mathbb{R}^K$2, 20 epochs of warmup followed by cosine decay, effective batch size $\mathbb{R}^K$3 over $\mathbb{R}^K$4A100 GPUs, PyTorch AMP, and global gradient clipping at norm $\mathbb{R}^K$5 (Li et al., 1 Oct 2025).

The paper reports ImageNet Top-1 results of $\mathbb{R}^K$6 and $\mathbb{R}^K$7 for ViT-Base + ADD variants, compared with $\mathbb{R}^K$8 for a standard ViT-Base, $\mathbb{R}^K$9 for a standard ViT-Large, and $$q(\mathbf{y}_t \mid \mathbf{y}_0) = \mathcal{N}\!\big(\mathbf{y}_t;\, \sqrt{\bar{\alpha}_t}\,\mathbf{y}_0,\,(1-\bar{\alpha}_t)\mathbf{I}\big).$$0 for a standard ViT-Huge at $$q(\mathbf{y}_t \mid \mathbf{y}_0) = \mathcal{N}\!\big(\mathbf{y}_t;\, \sqrt{\bar{\alpha}_t}\,\mathbf{y}_0,\,(1-\bar{\alpha}_t)\mathbf{I}\big).$$1 resolution. For MS COCO captioning, the reported CLIP Scores are $$q(\mathbf{y}_t \mid \mathbf{y}_0) = \mathcal{N}\!\big(\mathbf{y}_t;\, \sqrt{\bar{\alpha}_t}\,\mathbf{y}_0,\,(1-\bar{\alpha}_t)\mathbf{I}\big).$$2 for ground-truth captions, $$q(\mathbf{y}_t \mid \mathbf{y}_0) = \mathcal{N}\!\big(\mathbf{y}_t;\, \sqrt{\bar{\alpha}_t}\,\mathbf{y}_0,\,(1-\bar{\alpha}_t)\mathbf{I}\big).$$3 for shuffled captions, $$q(\mathbf{y}_t \mid \mathbf{y}_0) = \mathcal{N}\!\big(\mathbf{y}_t;\, \sqrt{\bar{\alpha}_t}\,\mathbf{y}_0,\,(1-\bar{\alpha}_t)\mathbf{I}\big).$$4 for PDD-generated captions, and $$q(\mathbf{y}_t \mid \mathbf{y}_0) = \mathcal{N}\!\big(\mathbf{y}_t;\, \sqrt{\bar{\alpha}_t}\,\mathbf{y}_0,\,(1-\bar{\alpha}_t)\mathbf{I}\big).$$5 for ADD-generated captions. The paper interprets these results as showing that the same one-hot diffusion backbone can serve both classification and multi-token generation (Li et al., 1 Oct 2025).

The broader ADD perspective has already appeared in task-specific settings. In speech enhancement, ADDSE uses absorbing discrete diffusion over neural codec codes, with a special absorbing state $$q(\mathbf{y}_t \mid \mathbf{y}_0) = \mathcal{N}\!\big(\mathbf{y}_t;\, \sqrt{\bar{\alpha}_t}\,\mathbf{y}_0,\,(1-\bar{\alpha}_t)\mathbf{I}\big).$$6, a denoising cross-entropy objective weighted by $$q(\mathbf{y}_t \mid \mathbf{y}_0) = \mathcal{N}\!\big(\mathbf{y}_t;\, \sqrt{\bar{\alpha}_t}\,\mathbf{y}_0,\,(1-\bar{\alpha}_t)\mathbf{I}\big).$$7, and an RQDiT architecture combining techniques from RQ-Transformer and diffusion Transformers for non-autoregressive modeling (Gonzalez, 25 Feb 2026). In MIMO detection, GD4 performs denoising directly in the discrete symbol space and enables fast inference with one or a few denoising evaluations, in contrast to existing diffusion-based detectors that operate in a continuous relaxed space (Lu et al., 1 May 2026). In discrete diffusion LLMs, watermarking work describes an authenticity-oriented notion in which outputs are made authentically attributable through a distribution-preserving Gumbel-max scheme seeded by sequence index, yielding a detector with an exponentially decaying probability of false detection in the token sequence length (Bagchi et al., 3 Nov 2025).

The acronym itself is not fully stable across the literature. The following usages are explicitly documented:

Paper	Meaning of “ADD”	Relation
"Authentic Discrete Diffusion Model" (Li et al., 1 Oct 2025)	Authentic Discrete Diffusion	Direct one-hot-space framework
"Information-Theoretic Discrete Diffusion" (Jeon et al., 28 Oct 2025)	“Authentic” as exact likelihood matching	Broader theoretical criterion
"Add and Thin: Diffusion for Temporal Point Processes" (Lüdke et al., 2023)	No separate algorithm formally called “ADD”	“Add” refers to addition of Poisson events
"Adversarial Diffusion Distillation" (Sauer et al., 2023)	Adversarial Diffusion Distillation	Unrelated acronym

This nomenclatural variation matters because several papers use “authentic” to mean mathematically faithful to discrete structure, but not all uses of “ADD” refer to the same method. The direct one-hot framework of (Li et al., 1 Oct 2025), the information-theoretic likelihood interpretation of (Jeon et al., 28 Oct 2025), and the CTMC exactness criteria of (Noguerales et al., 6 Jul 2026) are nevertheless mutually reinforcing. Taken together, they describe ADD as a family of discrete diffusion methods that preserve core diffusion structure directly in discrete representations, align losses with exact likelihood or ELBO identities, and require consistent interpretation of reverse-process parameterizations during training and sampling.