Scalability of variable-length diffusion models beyond demonstrated domains

Determine whether variable-length diffusion models based on trans-dimensional jump diffusion, which approximate the addition of new elements during generation, scale effectively to state spaces more complex than those originally used in their demonstrations, assessing both modeling accuracy and practical performance under increased state-space complexity.

Background

The paper highlights that while diffusion and flow matching methods are effective for fixed-length continuous and discrete states, variable-length generation—especially in continuous or multimodal spaces—remains challenging. A notable prior approach, trans-dimensional jump diffusion, introduces elements via an approximation during generation.

The authors explicitly note uncertainty regarding how such variable-length diffusion models behave when extended to more complex state spaces than those used in the initial demonstrations, motivating a need to rigorously evaluate their scalability and fidelity in broader settings.

References

Limited attempts to develop variable-length diffusion models exist \citep{Campbell2023TransDimensionalGM}, but these rely on an approximation related to the addition of new elements, and it is unclear how they scale to more complex spaces than the ones upon which they were demonstrated.

— Branching Flows: Discrete, Continuous, and Manifold Flow Matching with Splits and Deletions  (2511.09465 - Nordlinder et al., 12 Nov 2025) in Introduction (Section 1)

Nevertheless, enabling discrete diffusion models to adaptively determine the appropriate generation length remains an open problem that is being actively explored in recent studies.

— DRRG: A Discrete Diffusion Framework for Radiology Report Generation  (2608.24105 - Zhoua et al., 25 Aug 2026) in Section 4.2, Analysis of Training and Generation Settings

These are not interchangeable baselines: the first group provides algorithms, not pretrained genomic checkpoints, while most genomic design systems target short, curated regulatory sequences using specialized conditioning, continuous relaxations, or rewards. Scaling them to heterogeneous 4,096-bp sequences at single-nucleotide resolution remains unestablished, and retraining them would introduce new tuning choices rather than a controlled substitution.

— When Genomic Masking Priors Fail to Transfer: Strong Variant Prediction, Weak Functional Generation  (2609.04861 - Hu et al., 4 Sep 2026) in Section 2, paragraph “Discrete genomic diffusion and comparison scope”