SwiftVideo: Few-Step Video Generation

• SwiftVideo is a distillation framework for few-step video generation that improves temporal stability and realism by aligning flow trajectories and sample distributions under aggressive step reduction.
• It employs a three-stage pipeline—continuous-time consistency distillation, adversarial distribution alignment, and trajectory alignment via synthetic preference pairs—to preserve ODE structure and enhance output quality.
• SwiftVideo demonstrates significant improvements on OpenVid-1M, reducing the 4-step FVD from 321.95 (teacher) to 118.89 while maintaining frame quality close to that of the original model.

Searching arXiv for SwiftVideo and closely related few-step video generation distillation work. Search query: all:SwiftVideo few-step video generation trajectory alignment SwiftVideo is a distillation framework for few-step video generation that targets the failure regime of diffusion-based and flow-based video models when inference is reduced from conventional multi-step ODE integration to $2$–$4$ Euler steps. In the formulation introduced by “SwiftVideo: A Unified Framework for Few-Step Video Generation through Trajectory-Distribution Alignment,” the framework combines continuous-time consistency distillation, adversarial distribution alignment to real videos, and cross-step trajectory alignment via DPO, with the explicit goal of preserving ODE structure while improving realism under aggressive step compression (Sun et al., 8 Aug 2025).

1. Problem setting and motivation

SwiftVideo is situated in the acceleration literature for flow-matching and diffusion-style video synthesis, where the central difficulty is that direct step reduction induces both visual degradation and temporal instability. The motivating example in the reported OpenVid-1M experiments is the pretrained Wan model: at $25$ steps it attains FVD $98.06$, but at $4$ steps it deteriorates to FVD $321.95$. The paper organizes prior few-step methods into two broad families. Trajectory-preserving methods, such as LCM- and PCM-style consistency distillation, remain relatively stable but often become blurry under very few steps. Distribution-matching methods, including DMD2- and OSV-style approaches, can sharpen outputs but may destabilize the underlying trajectory structure; representative $4$-step FVD numbers reported in the paper include $192.93$ for LCM, $198.56$ for PCM, $167.42$ for DMD2, and $4$0 for OSV$4$1 (Sun et al., 8 Aug 2025).

A recurrent misconception in this area is that either trajectory preservation or distribution matching is sufficient on its own. SwiftVideo is explicitly framed against that claim. Its premise is that few-step video synthesis fails for two different reasons at once: inaccurate low-step trajectories and imperfect sample distribution alignment. This suggests that stable acceleration requires joint control of path fidelity and terminal sample quality rather than a single-objective distillation criterion.

2. Framework composition

SwiftVideo is organized as a three-stage pipeline. First, Continuous-time Consistency Distillation (CCD) transfers the pretrained flow into a consistency-style student without relying on discrete neighboring ODE points. Second, Distribution Alignment (DA) adversarially aligns reconstructed samples with the real video distribution rather than the teacher distribution. Third, Trajectory Alignment (TA) uses synthetic preference pairs generated at different inference step counts to push low-step outputs toward higher-step outputs of the same distilled model (Sun et al., 8 Aug 2025).

Component	Function
CCD	Preserve accurate ODE trajectories without discrete-time solver error
DA	Align generated videos with real-video distribution
TA	Align low-step trajectories with higher-step trajectories via DPO

The framework is demonstrated on Wan2.1-FUN-inp-480p-1.3B in an image-to-video setting on OpenVid-1M, with the student using the same backbone architecture as the teacher. No architectural replacement of the core DiT-style video transformer is required; the additional machinery is training-side, consisting primarily of the continuous-time distillation objective, a DINOv2-based discriminator for DA, and LoRA-based fine-tuning for TA (Sun et al., 8 Aug 2025).

3. Continuous-time consistency distillation

SwiftVideo assumes a rectified-flow or flow-matching generative process. Given a data sample $4$2 and a noise sample $4$3, the interpolation is

$4$4

with velocity

$4$5

The pretrained model defines the ODE

$4$6

SwiftVideo parameterizes the consistency model in a rectified-flow-compatible form,

$4$7

so that $4$8 and the mapping is directly compatible with Euler stepping. The resulting CCD objective is expressed as a squared loss over flow fields,

$4$9

where $25$0 approximates the continuous-time tangent term from the EMA teacher:

$25$1

Two stabilization devices are central. Tangent warmup increases $25$2 from $25$3 to $25$4 over the first $25$5 iterations, so training starts near a flow-matching distillation regime before transitioning to the full continuous-time objective. Tangent normalization rescales $25$6 as $25$7 with $25$8. For large DiT-based video backbones, the derivative $25$9 is computed via JVP, implemented with block-wise JVP and stop-gradient on the JVP term to control memory cost (Sun et al., 8 Aug 2025).

4. Distribution alignment and trajectory alignment

After CCD, the student is trajectory-faithful but still tends toward the characteristic blur of consistency-style models. DA addresses that limitation by reconstructing the clean sample as

$98.06$0

and then adversarially aligning $98.06$1 to real videos. The discriminator uses a frozen DINOv2 encoder with spatial heads for intra-frame quality and temporal heads for inter-frame coherence. The generator objective is

$98.06$2

with $98.06$3, while the discriminator uses the corresponding hinge loss on real $98.06$4 and fake $98.06$5. A key design choice is that DA targets the real data distribution rather than the teacher distribution, which is intended to avoid the teacher ceiling (Sun et al., 8 Aug 2025).

TA then addresses the residual gap between lower-step and higher-step inference within the same distilled model. SwiftVideo constructs a synthetic preference dataset $98.06$6, where $98.06$7 is a higher-step sample and $98.06$8 is a lower-step sample under the same conditioning $98.06$9. The paper adopts a Flow-DPO loss over velocity errors and adds a reflow regularizer on the preferred sample,

$4$0

with $4$1 in $4$2 and $4$3. The paper reports that DPO alone can degenerate by worsening dispreferred samples more than it improves preferred ones; the reflow term is introduced specifically to prevent that failure mode (Sun et al., 8 Aug 2025).

5. Training protocol and empirical performance

The reported training setup uses OpenVid-1M as the primary corpus, with $4$4 million videos selected randomly for training and $4$5 test videos for evaluation. CCD and DA are trained at resolution $4$6 with $4$7 frames, batch size $4$8, AdamW learning rate $4$9, and EMA rate $321.95$0. Tangent warmup lasts $321.95$1 iterations, DA begins after a warmup of $321.95$2 CCD iterations, and the time sampler is a log-normal $321.95$3, which the ablation identifies as the strongest among the tested choices. TA is then run as LoRA fine-tuning for about $321.95$4 iterations on a synthetic preference dataset of size $321.95$5 (Sun et al., 8 Aug 2025).

Selected OpenVid-1M FVD results reported in the paper are as follows.

Model / setting	FVD
Teacher Wan 25-step	98.06
Teacher Wan 4-step	321.95
LCM 4-step	192.93
OSV$321.95$6 4-step	146.59
SwiftVideo 4-step	118.89
SwiftVideo 2-step	199.61

The ablation trajectory is similarly structured. DA-only yields FVD $321.95$7, DCD+DA yields $321.95$8, CCD+DA yields $321.95$9, and adding TA reduces FVD further to $4$0. On VBench at $4$1 steps, SwiftVideo reports Temporal Quality $4$2, Frame Quality $4$3, I2V Subject $4$4, and I2V Background $4$5, with Frame Quality very close to the teacher’s $4$6. On VBench-I2V, the teacher’s Frame Quality is $4$7 and SwiftVideo at $4$8 steps reaches $4$9 (Sun et al., 8 Aug 2025).

6. Position in accelerated video generation and limitations

SwiftVideo belongs to the distillation branch of video generation acceleration rather than to architectural linearization or hardware-specialized redesign. This distinguishes it from FlashVideo, which replaces quadratic autoregressive attention with RetNet and reports a reduction of inference complexity from $192.93$0 to $192.93$1 together with a $192.93$2 efficiency improvement over a traditional autoregressive-based transformer (Lei et al., 2023). It is also distinct from SnapGen-V, which pursues on-device diffusion acceleration through a compact $192.93$3B latent UNet, hardware-aware temporal-layer search, adversarial fine-tuning, and $192.93$4-step sampling, reporting generation of a $192.93$5-second video on an iPhone 16 PM within $192.93$6 seconds (Wu et al., 2024). SwiftVideo instead assumes a pretrained flow-based teacher and asks how far few-step distillation can be pushed without catastrophic collapse (Sun et al., 8 Aug 2025).

The limitations reported for SwiftVideo are largely consequences of that choice. CCD requires JVP computation and the full training pipeline comprises three stages, including adversarial alignment and LoRA-based preference fine-tuning. The empirical study is confined to $192.93$7, $192.93$8-frame image-to-video generation rather than higher-resolution or substantially longer settings. The paper also states that there remains a quality gap to the fully many-step regime, and that robustness beyond OpenVid-1M and beyond the tested temporal horizon is not yet established. Future directions identified in the work include scaling to larger and longer video models, moving toward one-step generation, extending beyond image-to-video, and clarifying the relation between average and instantaneous velocity in continuous-time consistency distillation (Sun et al., 8 Aug 2025).