Performance of continuous diffusion models applied to discrete data versus discrete diffusion models

Determine whether continuous diffusion models trained on continuous relaxations of discrete data can match the generative performance of discrete diffusion models on the same discrete-data tasks under comparable training and evaluation conditions.

Background

Recent work has applied flow-map or consistency-based (continuous) diffusion distillation techniques to problems with inherently discrete data by lifting the data into a continuous space. While these approaches show promise, it is not yet established whether operating in continuous space on discrete data achieves performance competitive with native discrete diffusion models.

This question is important for guiding future method development and benchmarking: if continuous-on-discrete methods can match discrete diffusion models, they may offer practical benefits from the rich continuous-diffusion distillation toolbox; if not, research should focus on improving discrete-native approaches.

References

Currently, it remains to be seen whether these continuous models on discrete data can match the performance of discrete diffusion models.

— Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMD  (2603.20155 - Hoogeboom et al., 20 Mar 2026) in Section 4 (Related work), Deterministic Diffusion Distillation

Our results serve as a starting point for what this paper leaves open: few/one-step generation (given ELF trained on raw T5Gemma-2 under-integrates at low NFE), practical-scale models for real-world tasks such as QA, and adapting existing AR models directly into embeddings for DLMs (T5Gemma-2 is adapted from Gemma-3~\citep{team2025gemma}, and one would want to skip that step).

— Scaling and Distilling Text Embeddings for Better Diffusibility  (2610.01016 - Zhang et al., 1 Oct 2026) in Conclusion