Continuous Diffusion Decoders

• Continuous diffusion decoders are architectures defined by continuous-state or continuous-time denoising dynamics for accurate data reconstruction and generation.
• They use various methods, including ODE/SDE solvers, discrete-step Gaussian chains, and hybrid continuous–discrete models, to balance reconstruction fidelity and computational efficiency.
• Applications span language modeling, image and video processing, and communication coding, each leveraging continuous trajectories for improved token readability and performance.

Searching arXiv for relevant papers on continuous diffusion decoders and closely related decoder architectures. Continuous diffusion decoders are decoder architectures in which reconstruction or generation proceeds through denoising dynamics in a continuous state space and, in some cases, continuous time. The term does not denote a single canonical construction. In recent literature it spans continuous-time ODE or SDE decoders, discrete-step Gaussian chains over continuous latents, hybrid continuous–discrete token decoders, one-step surrogates distilled from multi-step reverse processes, and decoders that read out intermediate states of an ongoing diffusion trajectory. Their role also varies by domain: some are direct generators, some are conditional reconstruction modules, and some are communication or coding decoders whose objective is faithful inversion rather than open-ended sample diversity (Dieleman et al., 2022, Fu et al., 12 Jan 2026, Zhang et al., 17 May 2026, Chen et al., 7 Aug 2025).

1. Definition and scope

The recent literature uses the label in several non-equivalent ways. For expository convenience, the following categories are useful as Editor’s terms. They summarize distinctions that are explicit in the cited works.

Class	Representative systems	Defining property
Continuous-time decoder	CDCD (Dieleman et al., 2022), LTT (Fu et al., 12 Jan 2026)	Reverse dynamics expressed as an ODE or SDE in continuous time
Continuous-state, discrete-step decoder	DDECC (Choukroun et al., 2022), VCDC (Zhang et al., 17 May 2026), qLDPC continuous DF (Liu et al., 26 Sep 2025)	Latent state is continuous-valued, but sampling is a finite-step chain
Hybrid continuous–discrete decoder	CANDI (Pynadath et al., 26 Oct 2025), CCDD (Zhou et al., 3 Oct 2025)	Continuous latent denoising is coupled to explicit discrete token or mask states
One-step or shortcut decoder	SODEC (Chen et al., 7 Aug 2025), ECCFM (Lei et al., 1 Dec 2025), multi-scale one-step distillation (Wang et al., 20 Mar 2026)	Multi-step reverse diffusion is replaced by one-step or few-step decoding
Trajectory-readout decoder	DiffusionBrowser (Hong et al., 15 Dec 2025)	Intermediate denoising states are decoded into previews rather than only the final sample
Boundary case	Cola DLM (Guo et al., 7 May 2026)	Diffusion models a latent prior, while the final decoder is separate and non-diffusive

This multiplicity of meanings matters because several papers are careful to distinguish a decoder that is merely continuous-valued from one that is genuinely continuous-time. The channel decoder of VCDC, for example, operates over continuous LLR vectors but is explicitly described as a finite-step denoising chain rather than a continuous-time diffusion decoder (Zhang et al., 17 May 2026). Conversely, CDCD is presented as continuous both in time and input space for categorical data (Dieleman et al., 2022).

A second scope distinction concerns purpose. In image tokenization and compression, diffusion decoders are often conditional reconstruction modules, not downstream content generators. The acceleration paper on image tokenization states that such decoders are dedicated to faithful reconstruction rather than content generation, and SODEC makes a related point by arguing that a sufficiently informative compression latent can make multi-step refinement unnecessary (Wang et al., 20 Mar 2026, Chen et al., 7 Aug 2025).

2. Degrees of continuity and mathematical formulations

A continuous diffusion decoder may be continuous in state, in time, or in both. The strongest version appears in CDCD, where categorical data are represented by learned continuous embeddings, corrupted by Gaussian noise, and denoised through a continuous-time probability-flow ODE. Its forward corruption is written as

$$\mathbf{x}_t = \mathbf{x}_0 + t \boldsymbol{\epsilon}, \qquad \boldsymbol{\epsilon}\sim\mathcal N(0,I),$$

and the score is induced from token posteriors through

$$\hat{\mathbf{s}}(\mathbf{x},t)=\frac{\hat{\mathbf{x}}_0-\mathbf{x}}{t^2}.$$

Generation therefore remains continuous until final discretization (Dieleman et al., 2022).

A weaker but common form is continuous-state decoding with discrete time indices. NeoDiff remains in embedding space but introduces token-specific intrinsic time $\tau_t$, with forward kernel

$$q(\mathbf{z}_t\mid \mathbf{z}_0,\tau_t)=\mathcal N\!\left(\sqrt{\bar{\alpha}(\tau_t)}\,\mathbf{z}_0,\bar{\beta}(\tau_t)I\right),$$

and a Poisson process over tokenwise diffusion progress. The result is a non-simultaneous continuous decoder: latent states are continuous, but different tokens occupy different corruption stages (Li et al., 28 May 2025).

The same separation appears in communication decoders. VCDC translates AWGN corruption into LLR space and treats the latent state as a continuous Gaussian vector,

$$q(\mathbf{z}_s\mid \mathbf{x})=\mathcal N\!\left(\frac{2}{w_s^2}\mathbf{x},\frac{4}{w_s^2}I\right),$$

yet the reverse process traverses a discrete set of SNR-indexed levels rather than a continuous-time SDE or ODE (Zhang et al., 17 May 2026). DDECC likewise uses an unscaled additive Gaussian forward chain over continuous received words, but remains a discrete Markov chain with finite reverse iterations (Choukroun et al., 2022).

A third formulation replaces reverse diffusion entirely by deterministic transport. LTT constructs a Gaussian smoothing path

$$X_t\mid X_1=x_1 \sim \mathcal N(x_1,\sigma(t)^2 I_d)$$

and learns a velocity field by conditional flow matching, with inference given by

$\frac{d}{dt}X_t=v_\theta(X_t,t).$

This is a continuous-time generative decoder in the ODE sense, but it is trained by velocity regression rather than explicit score matching (Fu et al., 12 Jan 2026).

Hybrid models treat continuity as one part of a larger decoder state. CANDI uses masked discrete corruption to preserve explicit token anchors while applying Gaussian corruption to the corrupted subset, and CCDD places a discrete token state $x_t$ and a continuous representation state $z_t$ in a joint reverse process

$$p_\theta(x_s,z_s\mid x_t,z_t)=p_\theta^{\mathrm{disc}}(x_s\mid x_t,z_t)\,p_\theta^{\mathrm{cont}}(z_s\mid x_t,z_t).$$

In both cases, the continuous component is used to recover global geometry while the discrete component stabilizes decoding back to tokens (Pynadath et al., 26 Oct 2025, Zhou et al., 3 Oct 2025).

3. Language and sequence decoders

Language modeling has become the main arena in which the strengths and weaknesses of continuous diffusion decoders are analyzed directly. CDCD is the clearest direct construction: it diffuses token embeddings in continuous time and continuous input space, predicts token posteriors from noisy embeddings, and uses those posteriors to reconstruct the score field needed by the reverse ODE (Dieleman et al., 2022). This design preserves continuous uncertainty throughout denoising, but the final token readout remains a central difficulty.

NeoDiff modifies this setting by arguing that standard continuous text diffusion applies a single global timestep to all tokens, which is a poor inductive bias for decoding. Its bi-temporal formulation separates extrinsic sentence-level time $$\hat{\mathbf{s}}(\mathbf{x},t)=\frac{\hat{\mathbf{x}}_0-\mathbf{x}}{t^2}.$$0 from intrinsic token-level time $$\hat{\mathbf{s}}(\mathbf{x},t)=\frac{\hat{\mathbf{x}}_0-\mathbf{x}}{t^2}.$$1, sampled through a Poisson process and predicted in reverse by a context-aware time predictor. This makes the decoder token-adaptive while remaining embedding-space and non-autoregressive (Li et al., 28 May 2025).

CANDI goes further by diagnosing a specific failure mode of pure Gaussian diffusion on discrete data: token identifiability. It shows that discrete identity corruption scales with vocabulary size in a way that continuous rank degradation does not, producing what the paper calls temporal dissonance. Its solution is a hybrid decoder in which a masking process controls which token identities remain clean while continuous Gaussian noise acts only on corrupted positions. The model therefore preserves explicit discrete anchors for conditional learning while still obtaining a continuous score field for joint refinement and simple classifier guidance (Pynadath et al., 26 Oct 2025).

CCDD makes a closely related argument but frames it as a decoder bottleneck rather than only a corruption mismatch. The paper claims that continuous diffusion is more expressive than discrete diffusion and looped transformers, but that pure continuous models underperform because decoding tokens from rich continuous trajectories is hard in practice. Its remedy is a single denoiser with two coupled heads, one for token logits and one for continuous latent denoising. On LM1B, a 92.1M-parameter CCDD-MDiT with Qwen3 embeddings reports validation perplexity $$\hat{\mathbf{s}}(\mathbf{x},t)=\frac{\hat{\mathbf{x}}_0-\mathbf{x}}{t^2}.$$2 versus $$\hat{\mathbf{s}}(\mathbf{x},t)=\frac{\hat{\mathbf{x}}_0-\mathbf{x}}{t^2}.$$3 for a same-scale MDLM reimplementation, and CCDD-MMDiT reports $$\hat{\mathbf{s}}(\mathbf{x},t)=\frac{\hat{\mathbf{x}}_0-\mathbf{x}}{t^2}.$$4 (Zhou et al., 3 Oct 2025).

Not every latent diffusion LLM is, however, a diffusion decoder in the direct sense. Cola DLM explicitly separates a continuous latent prior from a separate conditional text decoder. Its diffusion component performs latent prior transport rather than token-level observation recovery, and the paper stresses that it should be understood as a hierarchical latent diffusion LLM rather than a direct continuous diffusion decoder (Guo et al., 7 May 2026). This boundary case has become important because it shows that some of the apparent gains of “continuous diffusion for text” may actually come from moving diffusion to a semantic prior while leaving final text realization to another decoder.

4. Image, video, and multimodal decoders

In vision, continuous diffusion decoders are often reconstruction modules conditioned on a shared latent. MDDVAE replaces the usual feed-forward image decoder in a multimodal VAE with a conditional DDPM-style decoder, while retaining ordinary VAE decoders for simpler modalities. The paper is explicit that this substitution may no longer yield a strict ELBO, but it reports major FID gains: on CUB, MDDVAE obtains image FID $$\hat{\mathbf{s}}(\mathbf{x},t)=\frac{\hat{\mathbf{x}}_0-\mathbf{x}}{t^2}.$$5 for text-to-image and $$\hat{\mathbf{s}}(\mathbf{x},t)=\frac{\hat{\mathbf{x}}_0-\mathbf{x}}{t^2}.$$6 for unconditional generation, compared with $$\hat{\mathbf{s}}(\mathbf{x},t)=\frac{\hat{\mathbf{x}}_0-\mathbf{x}}{t^2}.$$7 and $$\hat{\mathbf{s}}(\mathbf{x},t)=\frac{\hat{\mathbf{x}}_0-\mathbf{x}}{t^2}.$$8 for MLD; on CelebAMask-HQ it reports FID $$\hat{\mathbf{s}}(\mathbf{x},t)=\frac{\hat{\mathbf{x}}_0-\mathbf{x}}{t^2}.$$9 in several conditioning settings (Wesego et al., 2024).

Compression papers shift the emphasis from expressivity to latency. SODEC argues that in image compression the decoder already receives a sufficiently informative latent, so iterative reverse diffusion is unnecessary. It uses a pre-trained VAE-based front end, a single UNet denoising evaluation at fixed timestep $\tau_t$0, and a fidelity-guidance module derived from an auxiliary decoder. On $\tau_t$1 images with an A6000 GPU, the paper reports total inference time $\tau_t$2 ms, with decoding time $\tau_t$3 ms, and summarizes the latency gain over earlier diffusion codecs as more than $\tau_t$4 (Chen et al., 7 Aug 2025).

The acceleration paper on image tokenization addresses the same bottleneck from a different direction. It introduces a multi-scale sampling strategy in which decoding begins at a coarse resolution and doubles the resolution at each stage, claiming a theoretical speedup of $\tau_t$5 relative to standard full-resolution sampling, and then distills each scale into a single-step denoising model. The stated outcome is an order-of-magnitude reduction in decoding time with little degradation in output quality (Wang et al., 20 Mar 2026).

Video work has broadened the notion of decoder still further. DiffusionBrowser is a lightweight decoder attached to intermediate video-diffusion features $\tau_t$6, where $\tau_t$7 is denoising timestep and $\tau_t$8 is transformer block. It predicts RGB and scene intrinsics such as depth, normals, metallicity, and roughness, yielding previews at arbitrary sampled timesteps or blocks. The paper reports preview generation in less than $\tau_t$9 second for a $$q(\mathbf{z}_t\mid \mathbf{z}_0,\tau_t)=\mathcal N\!\left(\sqrt{\bar{\alpha}(\tau_t)}\,\mathbf{z}_0,\bar{\beta}(\tau_t)I\right),$$0-second video, a runtime of $$q(\mathbf{z}_t\mid \mathbf{z}_0,\tau_t)=\mathcal N\!\left(\sqrt{\bar{\alpha}(\tau_t)}\,\mathbf{z}_0,\bar{\beta}(\tau_t)I\right),$$1 s at $$q(\mathbf{z}_t\mid \mathbf{z}_0,\tau_t)=\mathcal N\!\left(\sqrt{\bar{\alpha}(\tau_t)}\,\mathbf{z}_0,\bar{\beta}(\tau_t)I\right),$$2 of denoising steps, and more than $$q(\mathbf{z}_t\mid \mathbf{z}_0,\tau_t)=\mathcal N\!\left(\sqrt{\bar{\alpha}(\tau_t)}\,\mathbf{z}_0,\bar{\beta}(\tau_t)I\right),$$3 real-time speed (Hong et al., 15 Dec 2025). This is continuous only in an interactional sense: the decoder exposes the denoising trajectory, but its support is over discrete sampled states rather than an explicit decoder field over all $$q(\mathbf{z}_t\mid \mathbf{z}_0,\tau_t)=\mathcal N\!\left(\sqrt{\bar{\alpha}(\tau_t)}\,\mathbf{z}_0,\bar{\beta}(\tau_t)I\right),$$4.

5. Communication and coding decoders

Coding theory has produced some of the clearest examples of continuous-state diffusion decoders. DDECC models AWGN corruption of a BPSK-modulated codeword as an additive Gaussian diffusion process

$$q(\mathbf{z}_t\mid \mathbf{z}_0,\tau_t)=\mathcal N\!\left(\sqrt{\bar{\alpha}(\tau_t)}\,\mathbf{z}_0,\bar{\beta}(\tau_t)I\right),$$5

then learns a reverse denoiser conditioned on the number of parity-check errors and augments it with syndrome-based line search. The state is continuous-valued, but the process is explicitly discrete-time and the network ultimately predicts a binary multiplicative corruption pattern rather than a continuous score (Choukroun et al., 2022).

VCDC replaces DDECC’s direct denoising over received words with a variational-diffusion decoder in LLR space. The diffusion state is a continuous LLR vector, the reverse kernel is analytically inherited from the VDM/AWGN model, and the denoiser is a BP-structured neural module. Inference omits reverse noise injection and caps the number of reverse steps at $$q(\mathbf{z}_t\mid \mathbf{z}_0,\tau_t)=\mathcal N\!\left(\sqrt{\bar{\alpha}(\tau_t)}\,\mathbf{z}_0,\bar{\beta}(\tau_t)I\right),$$6, with early stopping based on parity-check errors. For LDPC $$q(\mathbf{z}_t\mid \mathbf{z}_0,\tau_t)=\mathcal N\!\left(\sqrt{\bar{\alpha}(\tau_t)}\,\mathbf{z}_0,\bar{\beta}(\tau_t)I\right),$$7, the paper reports for Ours-20 a complexity of $$q(\mathbf{z}_t\mid \mathbf{z}_0,\tau_t)=\mathcal N\!\left(\sqrt{\bar{\alpha}(\tau_t)}\,\mathbf{z}_0,\bar{\beta}(\tau_t)I\right),$$8 FLOPs and model size $$q(\mathbf{z}_t\mid \mathbf{z}_0,\tau_t)=\mathcal N\!\left(\sqrt{\bar{\alpha}(\tau_t)}\,\mathbf{z}_0,\bar{\beta}(\tau_t)I\right),$$9 B, versus $$q(\mathbf{z}_s\mid \mathbf{x})=\mathcal N\!\left(\frac{2}{w_s^2}\mathbf{x},\frac{4}{w_s^2}I\right),$$0 FLOPs and $$q(\mathbf{z}_s\mid \mathbf{x})=\mathcal N\!\left(\frac{2}{w_s^2}\mathbf{x},\frac{4}{w_s^2}I\right),$$1 B for BP, $$q(\mathbf{z}_s\mid \mathbf{x})=\mathcal N\!\left(\frac{2}{w_s^2}\mathbf{x},\frac{4}{w_s^2}I\right),$$2 FLOPs and $$q(\mathbf{z}_s\mid \mathbf{x})=\mathcal N\!\left(\frac{2}{w_s^2}\mathbf{x},\frac{4}{w_s^2}I\right),$$3 MB for HGN, and $$q(\mathbf{z}_s\mid \mathbf{x})=\mathcal N\!\left(\frac{2}{w_s^2}\mathbf{x},\frac{4}{w_s^2}I\right),$$4 FLOPs and $$q(\mathbf{z}_s\mid \mathbf{x})=\mathcal N\!\left(\frac{2}{w_s^2}\mathbf{x},\frac{4}{w_s^2}I\right),$$5 KB for DDECC-Max (Zhang et al., 17 May 2026).

ECCFM takes the next step and collapses the reverse diffusion trajectory into one-step decoding. It casts reverse denoising as a PF-ODE consistency problem and trains a decoder $$q(\mathbf{z}_s\mid \mathbf{x})=\mathcal N\!\left(\frac{2}{w_s^2}\mathbf{x},\frac{4}{w_s^2}I\right),$$6 that is constant along the trajectory. The ECC-specific contribution is a differentiable soft-syndrome time condition $$q(\mathbf{z}_s\mid \mathbf{x})=\mathcal N\!\left(\frac{2}{w_s^2}\mathbf{x},\frac{4}{w_s^2}I\right),$$7, which replaces the discontinuous hard syndrome count used in earlier decoders. The paper reports $$q(\mathbf{z}_s\mid \mathbf{x})=\mathcal N\!\left(\frac{2}{w_s^2}\mathbf{x},\frac{4}{w_s^2}I\right),$$8 speedup over DDECC on short codes and $$q(\mathbf{z}_s\mid \mathbf{x})=\mathcal N\!\left(\frac{2}{w_s^2}\mathbf{x},\frac{4}{w_s^2}I\right),$$9 on medium-to-long codes, while achieving the best or second-best $$X_t\mid X_1=x_1 \sim \mathcal N(x_1,\sigma(t)^2 I_d)$$0 on many BCH, Polar, LDPC, MacKay, and CCSDS benchmarks (Lei et al., 1 Dec 2025).

Wireless image transmission has generated a different line of work. LTT interprets the physical channel itself as a point on a continuous probability path and decodes by integrating a learned ODE from the channel-calibrated landing time to the clean endpoint. On DIV2K over AWGN at $$X_t\mid X_1=x_1 \sim \mathcal N(x_1,\sigma(t)^2 I_d)$$1 dB, it reports improvements over a diffusion-based generative baseline and DeepJSCC of $$X_t\mid X_1=x_1 \sim \mathcal N(x_1,\sigma(t)^2 I_d)$$2 and $$X_t\mid X_1=x_1 \sim \mathcal N(x_1,\sigma(t)^2 I_d)$$3 in PSNR, and $$X_t\mid X_1=x_1 \sim \mathcal N(x_1,\sigma(t)^2 I_d)$$4 and $$X_t\mid X_1=x_1 \sim \mathcal N(x_1,\sigma(t)^2 I_d)$$5 in MS-SSIM, using only $$X_t\mid X_1=x_1 \sim \mathcal N(x_1,\sigma(t)^2 I_d)$$6 ODE steps (Fu et al., 12 Jan 2026). By contrast, the AF–DDIM decoder for sub-THz multi-hop relays interprets the forward relay chain as a VP diffusion process but uses a matched discrete DDIM-style reverse schedule. It is therefore diffusion-inspired and channel-aware, yet not a continuous-time decoder in the score-based sense (Ercetin et al., 3 Jan 2026).

Quantum coding provides a useful counterexample. The qLDPC paper implements a continuous diffusion decoder for logical errors by shifting logical bits to $$X_t\mid X_1=x_1 \sim \mathcal N(x_1,\sigma(t)^2 I_d)$$7, adding Gaussian noise over $$X_t\mid X_1=x_1 \sim \mathcal N(x_1,\sigma(t)^2 I_d)$$8 steps, and learning a conditional reverse Gaussian chain on $$X_t\mid X_1=x_1 \sim \mathcal N(x_1,\sigma(t)^2 I_d)$$9. It reports that continuous diffusion achieves reasonably low LER on smaller code-capacity tasks, but masked diffusion decoders scale better than continuous diffusion decoders on larger bivariate bicycle codes (Liu et al., 26 Sep 2025). This is one of the clearest empirical demonstrations that continuous-state decoding is not automatically the strongest choice when the target space is intrinsically discrete.

6. Interface problems, acceleration, and persistent debates

A recurring debate is whether “continuous” means continuous-valued or continuous-time. Several papers insist on the distinction. VCDC is explicit that its diffusion state is continuous-valued but its algorithm is a finite-step denoising chain (Zhang et al., 17 May 2026). DiffusionBrowser says its previews are continuous in spirit and interaction design, yet discrete in implementation because they are attached to sampled timesteps and transformer blocks (Hong et al., 15 Dec 2025). SODEC, similarly, is a one-step latent diffusion decoder rather than a continuous reverse process (Chen et al., 7 Aug 2025).

A second debate concerns what counts as a decoder at all. Cola DLM keeps diffusion in a continuous latent prior and delegates text realization to a separate conditional decoder, so it is adjacent to rather than identical with a direct continuous diffusion decoder (Guo et al., 7 May 2026). The Diffusion Encoder makes a parallel point from the opposite side: it does not introduce a diffusion decoder, but its EM-inspired alternating scheme shows that once a diffusion process is inserted into one half of an encoder–decoder model, synchronization between latent inference and decoding becomes a central problem (Premkumar et al., 13 May 2026). This suggests that some of the practical difficulty attributed to “continuous diffusion decoding” is really a coupled training problem between trajectory model and readout model.

The decoder-interface view has become particularly influential in language modeling. The paper on continuous language diffusion as a decoder-interface problem argues that denoising succeeds when trajectories enter regions where the native decoder can read stable tokens, formalized as decoder basins. It reports that, on generated ELF states, frozen T5 token-embedding lookup recovers $\frac{d}{dt}X_t=v_\theta(X_t,t).$0–$\frac{d}{dt}X_t=v_\theta(X_t,t).$1 of native decoder decisions, a single linear readout reaches $\frac{d}{dt}X_t=v_\theta(X_t,t).$2 agreement at $\frac{d}{dt}X_t=v_\theta(X_t,t).$3k samples, and a conservative margin gate exits $\frac{d}{dt}X_t=v_\theta(X_t,t).$4–$\frac{d}{dt}X_t=v_\theta(X_t,t).$5 earlier in denoising steps (Du et al., 7 Jun 2026). The same work also argues that low mean-squared error can discard linguistic content, low perplexity can reflect low-entropy collapse, and clean latent reconstruction can coexist with a narrow decoder basin. The implication is that continuous diffusion decoders should be evaluated not only by denoising accuracy but by representation–decoder compatibility.

The accumulated evidence therefore points in two directions at once. One direction favors continuity: continuous-time ODE decoders, continuous latent trajectories, and hybrid continuous–discrete models all provide richer internal states, flexible numerical solvers, or stronger latent reasoning claims (Dieleman et al., 2022, Fu et al., 12 Jan 2026, Zhou et al., 3 Oct 2025). The other direction favors explicit anchoring: masked diffusion, one-step consistency maps, fidelity-guided decoders, and decoder-basin diagnostics repeatedly show that token realization, code validity, or source faithfulness often depend less on raw latent smoothness than on how the continuous state interfaces with a downstream readout (Liu et al., 26 Sep 2025, Lei et al., 1 Dec 2025, Chen et al., 7 Aug 2025, Du et al., 7 Jun 2026). A plausible implication is that future continuous diffusion decoders will be judged less by whether their latent path is continuous in a formal sense than by whether that path reaches a decoder-readable regime quickly, stably, and with domain-appropriate inductive structure.