Performance of continuous diffusion models applied to discrete data versus discrete diffusion models

Determine whether continuous diffusion models trained on continuous relaxations of discrete data can match the generative performance of discrete diffusion models on the same discrete-data tasks under comparable training and evaluation conditions.

Background

Recent work has applied flow-map or consistency-based (continuous) diffusion distillation techniques to problems with inherently discrete data by lifting the data into a continuous space. While these approaches show promise, it is not yet established whether operating in continuous space on discrete data achieves performance competitive with native discrete diffusion models.

This question is important for guiding future method development and benchmarking: if continuous-on-discrete methods can match discrete diffusion models, they may offer practical benefits from the rich continuous-diffusion distillation toolbox; if not, research should focus on improving discrete-native approaches.