Salt: Self-Consistent Distribution Matching with Cache-Aware Training for Fast Video Generation

Abstract: Distilling video generation models to extremely low inference budgets (e.g., 2--4 NFEs) is crucial for real-time deployment, yet remains challenging. Trajectory-style consistency distillation often becomes conservative under complex video dynamics, yielding an over-smoothed appearance and weak motion. Distribution matching distillation (DMD) can recover sharp, mode-seeking samples, but its local training signals do not explicitly regularize how denoising updates compose across timesteps, making composed rollouts prone to drift. To overcome this challenge, we propose Self-Consistent Distribution Matching Distillation (SC-DMD), which explicitly regularizes the endpoint-consistent composition of consecutive denoising updates. For real-time autoregressive video generation, we further treat the KV cache as a quality parameterized condition and propose Cache-Distribution-Aware training. This training scheme applies SC-DMD over multi-step rollouts and introduces a cache-conditioned feature alignment objective that steers low-quality outputs toward high-quality references. Across extensive experiments on both non-autoregressive backbones (e.g., Wan~2.1) and autoregressive real-time paradigms (e.g., Self Forcing), our method, dubbed \textbf{Salt}, consistently improves low-NFE video generation quality while remaining compatible with diverse KV-cache memory mechanisms. Source code will be released at \href{https://github.com/XingtongGe/Salt}{https://github.com/XingtongGe/Salt}.

• The paper introduces SC-DMD, a compositional regularizer that enforces endpoint consistency to overcome the compositionality deficit in denoising steps.
• It leverages cache-aware mixed-step training to align low- and high-quality features, thereby enhancing autoregressive video generation stability.
• Empirical results show significant improvements in imaging quality, semantic coherence, and long-horizon robustness over traditional DMD approaches.

Salt: Self-Consistent Distribution Matching with Cache-Aware Training for Fast Video Generation

Introduction and Motivation

Salt introduces a structurally principled framework for distilling video generation models to operate under extremely low inference budgets—enabling efficient, high-quality real-time video generation with as few as 2–4 denoising steps. Prevailing few-step distillation paradigms include trajectory-based consistency objectives and distribution-level matching approaches. Trajectory-level consistency often results in over-smoothed dynamics and conservative appearance due to ambiguity regressions inherent in complex video generative conditioning. Distribution Matching Distillation (DMD) remains mode-seeking, producing sharper samples. However, DMD’s per-step supervision lacks explicit constraints on multi-step composition, leading to "compositionality deficit": local denoising operators may be individually accurate but collectively incoherent when chained, manifesting as structural drift and degraded quality in multi-step rollouts (see Figure 1).

Figure 1: Compositionality deficit of DMD: increasing denoising steps with vanilla DMD distillation induces semantic and structural degradation in video generation.

Self-Consistent Distribution Matching Distillation (SC-DMD)

Salt resolves the compositionality deficit by augmenting DMD with a semigroup-defect regularizer—SC-DMD. This regularizer penalizes the deviation between directly and sequentially composed denoising updates (semigroup law adherence), enforcing endpoint-consistent denoising trajectories. The motivation is drawn from the mathematical structure of ODE flow maps, where composition law ensures stable transport across timesteps. SC-DMD maintains DMD’s sharpness-preserving distribution alignment, while ensuring local update composition is structurally coherent. The result is improved multi-step stability and higher fidelity across both non-autoregressive and autoregressive video generation regimes.

Figure 2: Comparison of training trajectories for few-step distillation methods; SC-DMD couples composition-related regularization with distribution matching for robust multi-step coherence.

Cache-Aware Mixed-Step Training for Autoregressive Generation

Autoregressive video generation introduces the additional complexity of the KV cache: each chunk is conditioned on prior model-generated context, whose quality degrades as errors accumulate. Training with a fixed step count $K$ introduces train–test mismatch due to variable cache quality during inference. Salt incorporates mixed-step rollouts ($K \in \{2,4,8\}$) in training, exposing the model to diverse cache-conditional regimes. Furthermore, cache-conditioned feature alignment is introduced: features generated under low-quality cache conditions are regularized toward those from higher-quality references, using token-wise relational margin penalties. This cross-step supervision, alongside mixed-step rollout, enhances long-horizon stability and extreme low-step quality.

Figure 3: Overview of Salt for autoregressive video generation: mixed-step rollout, SC-DMD endpoint regularization, and cache-conditioned feature alignment.

Empirical Results

Salt was evaluated across both non-autoregressive and autoregressive paradigms using Wan~2.1 as backbone. Metrics included VBench (I2V, T2V, and Long) scores for quality, semantic consistency, motion smoothness, and temporal flicker suppression.

Non-autoregressive Distillation:

Salt (SC-DMD) consistently outperformed DMD baselines under 4 NFE budgets, yielding higher Imaging Quality, Background Consistency, and Temporal Flickering (Table 1). The improvement is not achieved merely by denser training grids; explicit compositional regularization is essential. An adversarial variant (Ours-$\alpha$) produced further gains.

Autoregressive Real-Time Generation:

Salt’s training objective improved Total and Quality scores across LongLive, Self Forcing, and Causal Forcing backbones, with pronounced gains in semantic preservation. Notably, Salt’s 2-step variant outperformed baseline 4-step models, validating the robustness of compositional regularization in more aggressive few-step compression. Visualization demonstrates improved preservation of fine-grained textures and stability in high-dynamic scenes (Figure 4).

Figure 4: Qualitative comparison on texture-rich and high-dynamic scenes, showing improved fidelity and stable motion under Salt.

Cross-step consistency was also evaluated: while vanilla DMD exhibited strong structural drift across step counts, SC-DMD maintained robust object structure and semantic coherence for 2-, 4-, and 8-step sampling (Figure 5).

Figure 5: Cross-step consistency under identical seed and prompt; SC-DMD yields stable and coherent outputs regardless of inference step count.

Salt’s gains persist in long-horizon autoregressive settings (30s rollouts), demonstrating enhanced semantic stability and reduced drift (Table 2).

Mechanistic Analysis and Ablation

Displacement-normalized local semigroup defect quantifies the composition error of denoising operators on the test-time path. SC-DMD markedly reduced this defect (Figure 6), confirming improved local compositionality.

Figure 6: Displacement-normalized local semigroup defect shows SC-DMD yields more compositional denoising operators.

Ablations reveal SC-DMD’s effectiveness is contingent on mixed-step rollout in autoregressive settings; naive addition of self-consistency does not suffice. Reference alignment loss particularly aids low-budget (2-NFE) variants.

Implications and Future Directions

Salt establishes that resolving structural deficiencies in few-step video distillation—specifically, by promoting endpoint-consistent compositions—substantially advances the stability, quality, and semantic fidelity of generated videos. The cache-aware mixed-step training extends these improvements to autoregressive regimes with streaming rollouts, broadening applicability to real-time and interactive generation pipelines. The lightweight nature of SC-DMD regularization incurs negligible inference overhead and is agnostic to backbone architecture or memory mechanism.

Salt's approach—enforcing compositionality via semigroup-defect regularization while aligning low- and high-quality cache features—suggests future directions for structurally generalizable distillation objectives. Integrating semigroup-based constraints and distribution matching is promising for other domains where multi-step composition error propagates, such as sequential conditional generative modeling and streaming LLM inference. Further work may explore compositionality-preserving objectives for other modalities and leverage adaptive grid sampling and local error diagnostics to auto-calibrate step count in deployment.

Conclusion

Salt delivers a structurally rigorous training framework for few-step video diffusion distillation, combining self-consistent distribution matching and cache-aware mixed-step objectives. The method addresses the compositionality deficit in DMD by coupling endpoint-consistent regularization, preserves mode-seeking sharpness, and fortifies autoregressive rollouts against cache-induced drift. Extensive evaluations confirm consistent improvements in video generation quality, compositional stability, and semantic coherence across short and long horizons, establishing Salt as a robust paradigm for efficient real-time video synthesis (2604.03118).