Uniform Discrete Diffusion with Metric Path for Video Generation (2510.24717v1)

Abstract: Continuous-space video generation has advanced rapidly, while discrete approaches lag behind due to error accumulation and long-context inconsistency. In this work, we revisit discrete generative modeling and present Uniform discRete diffuSion with metric pAth (URSA), a simple yet powerful framework that bridges the gap with continuous approaches for the scalable video generation. At its core, URSA formulates the video generation task as an iterative global refinement of discrete spatiotemporal tokens. It integrates two key designs: a Linearized Metric Path and a Resolution-dependent Timestep Shifting mechanism. These designs enable URSA to scale efficiently to high-resolution image synthesis and long-duration video generation, while requiring significantly fewer inference steps. Additionally, we introduce an asynchronous temporal fine-tuning strategy that unifies versatile tasks within a single model, including interpolation and image-to-video generation. Extensive experiments on challenging video and image generation benchmarks demonstrate that URSA consistently outperforms existing discrete methods and achieves performance comparable to state-of-the-art continuous diffusion methods. Code and models are available at https://github.com/baaivision/URSA

• The paper presents URSA, which refines quantized spatiotemporal tokens via a metric-guided uniform discrete diffusion process for scalable video generation.
• It leverages discrete flow matching with a linearized metric path and resolution-dependent timestep shifting to balance local reconstruction with global coherence.
• Experimental results demonstrate URSA's competitive performance, surpassing discrete baselines with fewer inference steps and improved temporal consistency.

Uniform Discrete Diffusion with Metric Path for Video Generation

Introduction and Motivation

The paper introduces URSA, a framework for scalable video generation that leverages uniform discrete diffusion with a metric path. The motivation stems from the observed performance gap between continuous-space generative models—such as those based on diffusion in pixel or latent spaces—and discrete-space approaches that operate on quantized visual tokens. While continuous models have achieved high-fidelity synthesis and temporal coherence, discrete models have lagged due to error accumulation and poor long-context consistency, especially in video generation. URSA addresses these limitations by formulating video generation as an iterative global refinement of discrete spatiotemporal tokens, conceptually aligning discrete methods with continuous paradigms and substantially narrowing their performance gap.

Figure 1: Illustration of different image/video generation paradigms. Discrete-space approaches such as AR and MDM adopt non-refinable local generation, where produced tokens are fixed once generated. In contrast, URSA introduces iterative global refinement, conceptually aligning discrete methods with continuous-space approaches, and substantially narrowing their performance gap.

Methodology

Discrete Flow Matching and Metric Probability Path

URSA builds upon the discrete flow matching (DFM) framework, which models the evolution of data from an initial uniform categorical noise distribution to the target data distribution via a time-dependent probability path. The key innovation is the introduction of a linearized metric path, where the probability of transitioning between token states is governed by their embedding-space distance. This enables the model to perform global refinement, moving all tokens toward their target states in a manner that respects the hierarchical structure of visual data.

Figure 2: Global refinement via token distance in embedding space. Starting from categorical noise $x_0$ (left), the framework refines data based on token distance to get target data $x_1$ (right), enabling hierarchical structure generation from global semantics to fine details.

The metric path is parameterized by a scheduler $\beta_t = c \times (\frac{t}{1-t})^\alpha$, where $c$ and $\alpha$ are hyperparameters controlling the linearity between timestep $t$ and the average token distance. This design allows for fine-grained control over the perturbation process, which is critical for learning long-sequence data such as high-resolution images and videos.

Resolution-Dependent Timestep Shifting

To accommodate varying data resolutions, URSA introduces a resolution-dependent timestep shifting mechanism. The shifted timestep $\tilde{t} = \frac{t}{t + \lambda(1-t)}$ modulates the perturbation strength, with $\lambda > 1$ for higher resolutions to induce stronger perturbations and $\lambda < 1$ for lower resolutions for more gradual transitions. This mechanism ensures stable training and effective representation learning across diverse sequence lengths.

Asynchronous Timestep Scheduling

URSA further proposes an asynchronous temporal fine-tuning strategy, where each frame in a video sequence is assigned an independent noise level. This decoupling of noise schedules across frames enables the model to balance local frame reconstruction with global temporal coherence, supporting versatile tasks such as text-to-video, image-to-video, video interpolation, extrapolation, and start-end frame control within a unified architecture.

Training and Sampling Procedures

URSA employs a predictor model $p_\theta(\cdot|x_t)$, initialized from a pre-trained LLM (Qwen3), and trained using cross-entropy loss between ground-truth and predicted token sequences. Visual data is tokenized using the Cosmos or IBQ tokenizer, providing substantial compression and efficient representation. During training, timesteps are sampled uniformly for each frame, and the metric path is used to generate perturbed tokens. The model is trained on large-scale datasets (e.g., Koala-36M, DataComp, COYO) using 128 A100 GPUs.

Sampling is performed via an Euler solver, iteratively refining the token sequence along the velocity field induced by the metric path. The process is efficient, requiring significantly fewer inference steps compared to prior discrete models.

Experimental Results

Sampling Efficiency and Iterative Refinement

URSA demonstrates superior sampling efficiency and quality across both image and video generation tasks. Iterative global refinement is shown to be essential for maintaining temporal coherence and visual fidelity, especially in long video sequences.

Figure 3: Sampling performance across inference steps. Using the Cosmos tokenizer, image samples at $256 \times 256$ ($\sim$1K tokens) and video samples at $25 \times 384 \times 240$ ($\sim$10K tokens) are evaluated.

Path Linearity and Model Scaling

The linearized metric path yields strong linear correlation between timestep and token distance, which is critical for optimal model performance. Ablation studies reveal that increasing model size enhances semantic alignment but does not proportionally improve generation quality, indicating that tokenizer capacity is a bottleneck.

Figure 4: Sampling performance of different paths. Image samples at $256 \times 256$ are evaluated.

Figure 5: Sampling performance of different model sizes. All models are trained for the same epoch count and evaluated on $256 \times 256$ images and $25 \times 384 \times 240$ videos.

Timestep Conditioning and Shifting

Time-agnostic conditioning does not provide measurable benefits for uniform discrete diffusion, and excessive variance in timestep embedding can degrade performance. Resolution-dependent timestep shifting is shown to be effective for video generation, matching the performance of continuous models.

Figure 6: Model metrics across training iterations. $256 \times 256$ images are sampled for evaluation.

Figure 7: Timestep shifting across SNR schedules. $25 \times 384 \times 240$ videos are sampled for evaluation.

Zero-Shot Generalization and Control

URSA exhibits strong zero-shot generalization in video extrapolation and start-end frame control tasks, extending video length and maintaining spatial-temporal consistency without task-specific adaptation.

Figure 8: Zero-shot video extrapolation. The 4-second text-to-video result is extended to 40 seconds.

Figure 9: Zero-shot start-end frame control. The start-end frames are rendered with transparency.

Benchmark Performance

URSA achieves a VBench text-to-video score of 82.4, outperforming discrete and continuous baselines. In image-to-video generation, URSA reaches a VBench++ score of 86.2, on par with state-of-the-art open-source models. For text-to-image generation, URSA attains a DPG-Bench score of 86.0, exceeding previous discrete approaches. These results highlight URSA's ability to bridge the gap between discrete and continuous generative paradigms.

Implications and Future Directions

URSA's framework demonstrates that discrete generative models, when equipped with global iterative refinement and metric-guided probability paths, can achieve performance comparable to continuous diffusion models. The resolution-dependent timestep shifting and asynchronous scheduling strategies enable scalable and versatile video generation, supporting multi-task objectives within a single model. The findings suggest that further improvements in discrete tokenization—particularly in spatiotemporal compression and reconstruction quality—could unlock even higher fidelity and longer-context video synthesis.

Theoretically, URSA provides a unified perspective on discrete and continuous generative modeling, leveraging kinetic-optimal paths and embedding-space metrics to control the evolution of data distributions. Practically, the framework is well-suited for deployment in resource-constrained environments due to its sampling efficiency and reduced inference steps.

Future research should focus on advancing discrete tokenizers, exploring hybrid discrete-continuous architectures, and extending the metric path framework to other modalities and generative tasks. The integration of more expressive tokenization schemes and adaptive scheduling mechanisms may further enhance the scalability and generalization capabilities of discrete generative models.

Conclusion

URSA presents a uniform discrete diffusion framework with a metric path, introducing linearized global refinement and resolution-dependent timestep shifting to bridge the gap between discrete and continuous video generation paradigms. The asynchronous temporal scheduling strategy enables multi-task video generation in a unified model. Extensive experiments demonstrate that URSA consistently outperforms existing discrete methods and achieves competitive results with state-of-the-art continuous diffusion models. This work establishes a foundation for scalable, efficient, and versatile discrete generative modeling, with significant implications for future research in video synthesis and beyond.