Discrete Diffusion Samplers Overview

• Discrete diffusion samplers are algorithms that transform learned reverse-time models on discrete state spaces into structured outputs such as text, molecules, and image tokens.
• They encompass various families—including Euler, τ-leaping, and uniformization—that balance computational efficiency, approximation accuracy, and discretization error.
• Key challenges include managing error propagation and discretization bias, prompting developments in learnable, adaptive, and correction-based sampler designs.

Searching arXiv for the cited papers on discrete diffusion samplers and closely related sampler analyses. Discrete diffusion samplers are the algorithms used to run discrete-state generative diffusion and related flow models backward from a tractable prior or corrupted state to structured discrete outputs such as text, molecules, graphs, discrete image tokens, or masked sequences. Across the recent literature, the term covers several closely related settings: continuous-time Markov-chain samplers for discrete diffusion models, discrete-time approximations such as Euler and $\tau$-leaping, learned or distilled samplers for few-step inference, and, in adjacent continuous-state work, timestep-discretization strategies that determine how a fixed solver is deployed in the low-number-of-function-evaluations regime. A unifying view is that sampling quality depends not only on the learned denoiser or score estimator, but also on the sampler’s transition rule, timestep schedule, stochasticity, and numerical approximation error (Fu et al., 24 Sep 2025, Ren et al., 2024, Tang et al., 23 Feb 2026).

1. Continuous-time foundations and the meaning of a sampler

In discrete diffusion models, the forward noising process is typically formulated as a continuous-time Markov chain (CTMC) on a finite state space. A standard local form is

$$p_{t + \Delta t \mid t}(y \mid x) = \begin{cases} Q_t(x,y)\,\Delta t + o(\Delta t), & y \neq x, \ 1 + Q_t(x,x)\,\Delta t + o(\Delta t), & y = x, \end{cases}$$

where $Q_t$ is a transition-rate matrix (Fu et al., 24 Sep 2025). On product spaces such as $[S]^d$, tractable constructions often use token-wise independent forward dynamics and Hamming-neighbor transitions, so that one coordinate flips at a time (Srikanth et al., 12 Oct 2025).

The reverse process is another CTMC whose rates are tilted by likelihood ratios. A standard formula is

$$\tilde Q_t(x,y) = \begin{cases} \dfrac{p_t(y)}{p_t(x)}\, Q_t(y,x), & y \neq x, \ -\sum_{z\neq x} \tilde Q_t(x,z), & y = x, \end{cases}$$

or, in the notation of reverse-time discrete diffusion,

$$\overline Q_t(x,y) = \frac{p_t(y)}{p_t(x)} Q_t(y,x)$$

for $x\neq y$ (Fu et al., 24 Sep 2025, Srikanth et al., 12 Oct 2025, Ren et al., 2024). The discrete analogue of the continuous score is therefore a ratio such as

$$s_t(x)_{i,\hat x^i} = \frac{q_t(x^{\backslash i}\odot \hat x^i)}{q_t(x)},$$

which determines the reverse jump intensities (Srikanth et al., 12 Oct 2025).

A discrete diffusion sampler is any discrete-time approximation of this reverse CTMC. One chooses time points $T=t_0>t_1>\dots>t_M=\epsilon$ and iteratively samples from an approximate kernel $q_\theta(x_{t_{k+1}}\mid x_{t_k},t_k)$ until reaching near time $$p_{t + \Delta t \mid t}(y \mid x) = \begin{cases} Q_t(x,y)\,\Delta t + o(\Delta t), & y \neq x, \ 1 + Q_t(x,x)\,\Delta t + o(\Delta t), & y = x, \end{cases}$$0 (Fu et al., 24 Sep 2025). In this sense, Euler samplers, $$p_{t + \Delta t \mid t}(y \mid x) = \begin{cases} Q_t(x,y)\,\Delta t + o(\Delta t), & y \neq x, \ 1 + Q_t(x,x)\,\Delta t + o(\Delta t), & y = x, \end{cases}$$1-leaping, Tweedie $$p_{t + \Delta t \mid t}(y \mid x) = \begin{cases} Q_t(x,y)\,\Delta t + o(\Delta t), & y \neq x, \ 1 + Q_t(x,x)\,\Delta t + o(\Delta t), & y = x, \end{cases}$$2-leaping, predictor–corrector variants, uniformization-based samplers, and learned few-step samplers are all discrete diffusion samplers. This suggests that “sampler” refers not merely to an implementation detail, but to the numerical and probabilistic procedure that turns a learned reverse-time model into actual samples (Tang et al., 23 Feb 2026).

A parallel line of work on continuous-state diffusion and flow matching models emphasizes that sampling also depends on the timestep grid used to discretize an ODE or SDE solver. In that setting, the endpoint is a function of the time grid $$p_{t + \Delta t \mid t}(y \mid x) = \begin{cases} Q_t(x,y)\,\Delta t + o(\Delta t), & y \neq x, \ 1 + Q_t(x,x)\,\Delta t + o(\Delta t), & y = x, \end{cases}$$3, and “discrete diffusion samplers” can be interpreted as the pair consisting of a numerical solver and a discretization strategy (Yuan et al., 18 Mar 2026). A plausible implication is that, although continuous-state and discrete-state literatures differ in mathematical formalism, both center sampler design as a primary object rather than a purely secondary choice.

2. Canonical sampler families

Three sampler families recur throughout the literature. The first is the ancestral or Euler-type sampler, which uses a first-order approximation of the reverse jump dynamics. In discrete diffusion models, a representative local update is

$$p_{t + \Delta t \mid t}(y \mid x) = \begin{cases} Q_t(x,y)\,\Delta t + o(\Delta t), & y \neq x, \ 1 + Q_t(x,x)\,\Delta t + o(\Delta t), & y = x, \end{cases}$$4

applied factorized over coordinates (Fu et al., 24 Sep 2025). In the theory of discrete-state diffusion, Euler is also analyzed as a one-jump-per-coordinate approximation, with coordinate-wise categorical updates (Liang et al., 20 Sep 2025).

The second family is $$p_{t + \Delta t \mid t}(y \mid x) = \begin{cases} Q_t(x,y)\,\Delta t + o(\Delta t), & y \neq x, \ 1 + Q_t(x,x)\,\Delta t + o(\Delta t), & y = x, \end{cases}$$5-leaping, imported from chemical kinetics and CTMC simulation. Instead of allowing at most one jump per coordinate, it approximates the number of jumps over a finite interval by Poisson or Tweedie-distributed counts and applies them in parallel (Fu et al., 24 Sep 2025). In more formal treatments, $$p_{t + \Delta t \mid t}(y \mid x) = \begin{cases} Q_t(x,y)\,\Delta t + o(\Delta t), & y \neq x, \ 1 + Q_t(x,x)\,\Delta t + o(\Delta t), & y = x, \end{cases}$$6-leaping corresponds to freezing the reverse rate matrix over each interval and simulating a piecewise-constant CTMC (Ren et al., 2024, Liang et al., 20 Sep 2025). Tweedie $$p_{t + \Delta t \mid t}(y \mid x) = \begin{cases} Q_t(x,y)\,\Delta t + o(\Delta t), & y \neq x, \ 1 + Q_t(x,x)\,\Delta t + o(\Delta t), & y = x, \end{cases}$$7-leaping refines this by accounting for schedule changes through an analytic approximation of the reverse transition term (Fu et al., 24 Sep 2025, Liang et al., 20 Sep 2025).

The third family is uniformization. Here one upper-bounds the total outgoing rate by a dominating Poisson process, samples candidate jump times, and accepts true jumps with probabilities proportional to the actual reverse intensities. In the stochastic-integral analysis of discrete diffusion, uniformization is exact for the approximate reverse process and therefore incurs no time-discretization error; only truncation and score-approximation errors remain (Ren et al., 2024). Sample-complexity theory for discrete-state diffusion likewise assumes a uniformization-based sampler as the canonical reverse-time simulation mechanism (Srikanth et al., 12 Oct 2025).

These families differ in both computational structure and theoretical properties.

Sampler family	Core approximation	Noted property
Euler	First-order, one-jump-per-step approximation	Cheap per step; widely used (Fu et al., 24 Sep 2025, Liang et al., 20 Sep 2025)
$$p_{t + \Delta t \mid t}(y \mid x) = \begin{cases} Q_t(x,y)\,\Delta t + o(\Delta t), & y \neq x, \ 1 + Q_t(x,x)\,\Delta t + o(\Delta t), & y = x, \end{cases}$$8-leaping	Piecewise-constant CTMC with Poisson jump counts	Parallelizable; incurs discretization error (Ren et al., 2024)
Uniformization	Poisson thinning for exact simulation of approximate CTMC	No time-discretization error (Ren et al., 2024)

The literature also includes specialized discrete-time samplers such as JYS, $$p_{t + \Delta t \mid t}(y \mid x) = \begin{cases} Q_t(x,y)\,\Delta t + o(\Delta t), & y \neq x, \ 1 + Q_t(x,x)\,\Delta t + o(\Delta t), & y = x, \end{cases}$$9-RK2, and $Q_t$0-Trapezoidal for discrete diffusion models (Fu et al., 24 Sep 2025), as well as masked-diffusion-specific Euler and uniformization variants with explicit complexity analysis (Huang et al., 26 Sep 2025). In adjacent continuous-state work, classical solver families include DDIM, DPM-Solver, DPM-Solver++, iPNDM, and UniPC, with recent emphasis on schedule optimization and few-step deployment (Yuan et al., 18 Mar 2026).

3. Principal error sources and correctness questions

A central theme in recent work is that naive acceleration of discrete diffusion samplers fails for two distinct reasons. The first is compounding decoding error due to factorized predictions. The true reverse kernel is generally coupled across coordinates, whereas many practical samplers use factorized position-wise predictions; when large steps are taken, early mistakes alter later states and their scores, and the resulting errors cascade along the trajectory (Fu et al., 24 Sep 2025). This effect is especially visible on long sequences with strong dependencies, such as countdown-style synthetic tasks (Fu et al., 24 Sep 2025).

The second is discretization error from approximating the continuous-time reverse CTMC by a coarse discrete-time scheme. In the standard analysis of discrete diffusion models, first-order schemes have local truncation error $Q_t$1 and global error $Q_t$2, so large steps can badly misrepresent the true jump dynamics (Fu et al., 24 Sep 2025). More recent theory strengthens this picture by deriving non-asymptotic KL bounds for $Q_t$3-leaping, Euler, and Tweedie $Q_t$4-leaping using differential inequalities rather than Girsanov-style change-of-measure arguments (Liang et al., 20 Sep 2025).

For standard $Q_t$5-leaping, the new KL guarantees scale linearly with vocabulary size, improving over prior quadratic dependence, and they extend to Euler and Tweedie $Q_t$6-leaping as well (Liang et al., 20 Sep 2025). This suggests that sampler efficiency depends not only on empirical behavior but also on subtle rate-approximation properties that affect how discretization error accumulates over long horizons.

A distinct but related issue is sampler correctness. In discrete diffusion LLMs, learned denoiser error and sampler-induced error are often conflated by standard evaluation metrics. A sampler-centric oracle framework based on an exact hidden-Markov posterior shows that few-step discrete diffusion samplers are not distributionally correct even under an oracle denoiser: transition-level mismatch persists and vanishes only as the number of steps approaches the sequence length (Tang et al., 23 Feb 2026). Moreover, improvements in negative log-likelihood, generative perplexity, or MAUVE do not imply correct sampling (Tang et al., 23 Feb 2026).

This line of evidence complicates a common assumption that a better denoiser automatically yields a better sampler. Instead, the sampler itself can define a biased generative process. A plausible implication is that discrete diffusion models should be evaluated at both the model and sampler levels, particularly when the intended use requires calibrated sampling rather than only fluent or visually plausible outputs.

4. Learned, optimized, and instance-adaptive samplers

Recent work increasingly treats the sampler as a trainable object. In discrete diffusion models, “Learnable Sampler Distillation” introduces learnable coefficients $Q_t$7 that scale the model score inside the reverse update, so that a few-step student sampler aligns its intermediate score trajectory with that of a high-quality many-step teacher (Fu et al., 24 Sep 2025). The central objective is not final-sample matching, which is non-differentiable in categorical spaces, but alignment of differentiable intermediate score trajectories: $Q_t$8 with KL divergence used in practice (Fu et al., 24 Sep 2025). The relaxed training objective further perturbs the initial state within a small Hamming neighborhood, making the optimization easier and improving convergence (Fu et al., 24 Sep 2025).

LSD+ extends this idea by learning the time schedule itself through step sizes $Q_t$9, defining a non-uniform grid

$[S]^d$0

and matching effective transition terms rather than raw scores (Fu et al., 24 Sep 2025). This indicates that sampler coefficients and timestep placement are complementary design variables in discrete-state settings.

In continuous-state diffusion and flow matching, a closely related development is the optimization of timestep schedules for few-step solvers. Classical global schedules include uniform spacing and LogSNR spacing, while optimization-based global schedules include DMN, AYS, GITS, and LD3 (Yuan et al., 18 Mar 2026). “Few-Step Diffusion Sampling Through Instance-Aware Discretizations” argues that a single global timestep grid is suboptimal when trajectories differ substantially across samples or conditions (Yuan et al., 18 Mar 2026). It therefore introduces instance-aware discretization, in which a network

$[S]^d$1

predicts a per-instance schedule $[S]^d$2, consisting of base times, time shifts, and scale factors (Yuan et al., 18 Mar 2026). The method is trained by endpoint distillation against a high-NFE teacher and consistently improves few-step sampling across synthetic data, pixel-space diffusion, latent images, and video flow matching (Yuan et al., 18 Mar 2026).

Although instance-aware discretization is formulated for continuous-state models, it is explicitly framed as a contribution to the broader landscape of discrete diffusion samplers in the sense of timestep-schedule design (Yuan et al., 18 Mar 2026). This suggests that “discrete diffusion sampler” has two legitimate usages in the current literature: a discrete-state reverse CTMC sampler, and a discretized few-step solver for continuous-time diffusion dynamics.

5. Stochasticity, correction, and sampler refinement

Another major direction studies how stochasticity itself affects few-step inference. In discrete diffusion, highly deterministic transitions such as discrete probability flow converge rapidly in the low-NFE regime, but they suffer from error accumulation, whereas more stochastic transitions converge more slowly yet can attain higher final quality because they contract sampling errors (Yuan et al., 26 May 2026). The underlying mechanism is identified as an error-correcting effect induced by redundant transitions that symmetrically exchange mass between states (Yuan et al., 26 May 2026).

This analysis leads to “Discrete Churn and Restart Sampling” (DCRS), a training-free inference algorithm that alternates between forward and reverse diffusion processes to inject controlled stochasticity (Yuan et al., 26 May 2026). DCRS combines near-deterministic reverse updates with restart windows and local churning steps that reintroduce forward noise and then reverse it, thereby balancing efficiency and correction (Yuan et al., 26 May 2026). On image datasets, DCRS achieves up to a $[S]^d$3 reduction in sampling steps compared to standard samplers while maintaining competitive sample quality; on language benchmarks, the behavior is more nuanced and depends on the corruption process and sampling procedure (Yuan et al., 26 May 2026).

A related correction-oriented perspective appears in discrete flow models. “Corrected Samplers for Discrete Flow Models” reanalyzes $[S]^d$4-leaping and Euler under discrete flow matching and derives non-asymptotic discretization bounds without restrictions on transition rates and source distributions (Wan et al., 30 Jan 2026). By studying a one-step lower bound of the Euler sampler, it proposes time-corrected and location-corrected samplers, which reduce the discretization error of $[S]^d$5-leaping and Euler with almost no additional computational cost (Wan et al., 30 Jan 2026). The location-corrected sampler is shown to have lower iteration complexity than existing parallel samplers (Wan et al., 30 Jan 2026).

These developments suggest that stochasticity and correction are not merely auxiliary heuristics. They function as systematic mechanisms for modifying the rate at which local errors are accumulated, contracted, or redistributed across trajectories. This in turn connects discrete diffusion sampling to broader ideas in numerical analysis and controlled Markov processes.

6. Theory, applications, and practical implications

The theory of discrete diffusion samplers has become substantially more rigorous. A stochastic-integral framework based on Lévy-type Poisson random measures formulates discrete diffusion as a jump-process analogue of an Itô SDE and proves a Girsanov-type change-of-measure theorem for evolving intensities (Ren et al., 2024). Within that framework, the score-entropy loss is exactly the pathwise KL divergence between the true and approximate reverse chains (Ren et al., 2024). The same work gives the first KL error bound for $[S]^d$6-leaping and shows that uniformization is exact for the approximate reverse process (Ren et al., 2024).

A complementary theoretical program studies learning complexity. “Discrete State Diffusion Models: A Sample Complexity Perspective” provides the first sample-complexity guarantee for discrete-state diffusion, showing a $[S]^d$7 rate under a decomposition of score estimation error into approximation, statistical, optimization, and clipping components (Srikanth et al., 12 Oct 2025). Approximation error can vanish on finite domains with sufficiently wide networks, clipping error is dominated by the unbounded-score error, and the dominant contributions are statistical and optimization terms scaling as $[S]^d$8 (Srikanth et al., 12 Oct 2025). This provides a training-side counterpart to sampler-side convergence analyses.

Masked discrete diffusion has its own complexity theory. In masked text generation, Euler samplers can achieve $[S]^d$9-accuracy in total variation with $$\tilde Q_t(x,y) = \begin{cases} \dfrac{p_t(y)}{p_t(x)}\, Q_t(y,x), & y \neq x, \ -\sum_{z\neq x} \tilde Q_t(x,z), & y = x, \end{cases}$$0 discrete score evaluations, providing the first rigorous analysis of the typical Euler sampler in this setting (Huang et al., 26 Sep 2025). The same work introduces Mask-Aware Truncated Uniformization (MATU), which removes bounded-score assumptions and exploits the fact that each token can be unmasked at most once, attaining a nearly $$\tilde Q_t(x,y) = \begin{cases} \dfrac{p_t(y)}{p_t(x)}\, Q_t(y,x), & y \neq x, \ -\sum_{z\neq x} \tilde Q_t(x,z), & y = x, \end{cases}$$1-free complexity $$\tilde Q_t(x,y) = \begin{cases} \dfrac{p_t(y)}{p_t(x)}\, Q_t(y,x), & y \neq x, \ -\sum_{z\neq x} \tilde Q_t(x,z), & y = x, \end{cases}$$2 (Huang et al., 26 Sep 2025). This result indicates that the forward corruption process itself can qualitatively alter sampler complexity.

Applications span text generation, image generation, video generation, scientific sampling, combinatorial optimization, and latent-space posterior inference. Learnable sampler distillation improves text perplexity on SEDD-small, SEDD-medium, and RADD backbones, improves CIFAR-10 and ImageNet discrete image generation, and reduces error rates on the countdown task (Fu et al., 24 Sep 2025). Off-policy discrete diffusion samplers with replay buffers and MCMC exploration improve mode coverage on Ising and Potts models, multimodal Gray-coded synthetic densities, and data-to-energy Schrödinger bridges, and they extend to posterior sampling in the discrete latent spaces of VQ-VAE image models (Carter et al., 5 Feb 2026). Adaptive destruction processes and off-policy training in related continuous-space diffusion samplers further reinforce the view that few-step samplers are better understood as trainable finite-horizon policies than as fixed discretizations of an idealized continuous process (Gritsaev et al., 2 Jun 2025, Berner et al., 10 Jan 2025).

A concise synthesis of the recent literature yields several durable conclusions. First, sampler design is a first-class problem: transition rule, timestep schedule, and stochasticity materially affect quality even with a fixed denoiser. Second, few-step acceleration is limited by both model error and numerical error; correcting one without the other is often insufficient. Third, exactness, speed, and robustness typically cannot be simultaneously maximized, so practical sampler design depends on the target regime: deterministic few-step generation, asymptotically correct sampling, or high-fidelity teacher imitation. Finally, the boundary between discrete-state diffusion samplers and discretized continuous-time samplers has become increasingly porous, with common tools now including schedule optimization, distillation, off-policy control, and correction mechanisms (Yuan et al., 18 Mar 2026, Fu et al., 24 Sep 2025, Yuan et al., 26 May 2026).