Accelerate SemiCat and related hybrid continuous diffusion methods

Develop acceleration techniques for SemiCat and related continuous and hybrid diffusion language models using the continuous modelling toolbox, and determine how such acceleration methods interact with the models’ design choices.

Background

The paper introduces SemiCat, a hybrid method combining continuous Gaussian evolution with stochastic discrete token reveals, and studies related designs including DoubleCat, one-hot flows, embedded flows, asynchronous schedules, and adaptive sampling. Although SemiCat reaches competitive performance with discrete diffusion, its stochastic reveals and piecewise-continuous trajectories make it difficult to apply standard continuous-generation acceleration techniques.

The authors explicitly identify acceleration with the continuous toolbox as future work. The unresolved issue includes both constructing effective acceleration procedures for these hybrid methods and understanding how acceleration mechanisms interact with choices such as self-conditioning, embedding geometry, discrete reveals, and noise schedules.