Papers
Topics
Authors
Recent
Search
2000 character limit reached

FlowC2S: Flowing from Current to Succeeding Frames for Fast and Memory-Efficient Video Continuation

Published 19 Apr 2026 in cs.CV | (2604.17625v1)

Abstract: This paper introduces a novel methodology for generating fast and memory-efficient video continuations. Our method, dubbed FlowC2S, fine-tunes a pre-trained text-to-video flow model to learn a vector field between the current and succeeding video chunks. Two design choices are key. First, we introduce inherent optimal couplings, utilizing temporally adjacent video chunks during training as a practical proxy for true optimal couplings, resulting in straighter flows. Second, we incorporate target inversion, injecting the inverted latent of the target chunk into the input representation to strengthen correspondences and improve visual fidelity. By flowing directly from current to succeeding frames, instead of the common combination of current frames with noise to generate a video continuation, we reduce the dimensionality of the model input by a factor of two. The proposed method, fine-tuned from LTXV and Wan, surpasses the state-of-the-art scores across quantitative evaluations with FID and FVD, with as few as five neural function evaluations.

Summary

  • The paper introduces FlowC2S, a novel flow-based method that directly maps current to future video frames without relying on noise injection.
  • The approach cuts input dimensionality by half, achieving competitive FID/FVD scores with only 5–10 neural function evaluations, enhancing efficiency.
  • Target inversion and inherent optimal couplings are key innovations that improve convergence, temporal coherence, and real-time applicability in AR/VR.

FlowC2S: Direct Flow Modeling for Efficient Video Continuation

Problem Context and Conventional Paradigms

Current video continuation models predominantly utilize diffusion-based or flow-based generative paradigms, where both input frames and random noise are supplied at inference to synthesize future video chunks. This setup, most notably instantiated in methods like LTXVCondition and CausVid, results in increased memory and computational overhead due to the doubled input dimensionality and a high number of neural function evaluations (NFEs) required for plausible results. The dependency on large input tensors and iterative denoising hinders their deployability on resource-constrained devices and limits their applicability in latency-critical scenarios such as AR/VR.

Figure 1

Figure 1

Figure 1: (a) Standard approaches combine input frames and noise for conditioning, while (b) FlowC2S removes noise, modeling direct flow from observed to future frames, thus enabling sampling solely from the current video chunk.

Method: FlowC2S and its Innovations

FlowC2S circumvents the inefficiencies of conventional conditional generative pipelines by directly modeling the vector field mapping current to succeeding video frame distributions. The core design choices are:

  • Direct Current-to-Succeeding Frame Flow: Rather than initializing with Gaussian noise, the model is conditioned solely on the observed video chunk, reducing data dimensionality at inference by 2× and minimizing the redundancy inherent in stochastic initializations.
  • Inherent Optimal Couplings (OC): Exploiting the strong temporal continuity between adjacent video chunks of the same source, FlowC2S approximates optimal transport couplings without incurring the expensive computations usually required in high-dimensional spaces. This empirically leads to straighter sample trajectories, improved convergence, and a reduction in the number of required NFEs for high-quality synthesis.

Figure 2

Figure 2

Figure 2: Optimal transport heatmaps reveal intrinsic alignment between temporally adjacent video chunks, confirming the suitability of intra-video pairing as proxy optimal couplings.

  • Target Inversion (TI): To mitigate the mismatch between the shifted generative objective (from Gaussian-to-data to current-to-future frame flows), FlowC2S injects the latent of the target chunk—obtained via RF-Solver inversion—into the input representation at training, with stochastic sampling (parameterized by a Bernoulli selector). This augmentation constrains the flow estimation, improving visual fidelity and temporal alignment especially in data regimes with structured motion.

Figure 3

Figure 3: Training ablatives demonstrate lowest loss and best FID/FVD when fine-tuning with both inherent OC and TI; scratch training struggles with convergence.

Figure 4

Figure 4: Visualizations of ablations: only fine-tuning with both inherent OC and TI yields sharp, temporally coherent outputs; other setups show artifacts and temporal drift.

Experimental Findings and Quantitative Analysis

Evaluation on OpenVid and NuScenes benchmarks show that FlowC2S, leveraging both LTXV and Wan pre-trained foundations, achieves FID and FVD scores on par with or surpassing established approaches, but with a reduction in NFE from 40 to 5 (for LTXV) or parity at even lower NFE (for Wan), and with reduced peak GPU memory due to the halved input. Ablation studies highlight several critical findings:

  • Fine-tuning with both OC and TI is essential: Networks initialized from scratch or omitting either inductive bias shows clear disadvantages in training stability, final loss, and visual output quality, with pronounced artifacts and incoherence.
  • OC supports efficient sampling: Fewer NFEs suffice for high fidelity when using inherent couplings, with 5–10 matches or outperforms the 40-NFE baseline without OC/TI.

Figure 5

Figure 5

Figure 5

Figure 5: 5–10 NFEs with OC+TI match or surpass 40-NFE baseline; FlowC2S generalizes to significantly longer continuations than seen in training.

  • Generalization to Long Continuations: Training on 17- or 41-frame chunks suffices for stable outputs up to 113 frames; beyond this range, some degradation is observed, suggesting the method's robustness but also pinpointing the boundaries of its current generalization.

Figure 6

Figure 6: FlowC2S generates visually coherent, temporally stable continuations up to 113 frames, despite training on shorter sequences.

  • Failure Regimes: For very long rollouts (>113 frames), degradation appears as interpolation artifacts and loss of temporal structure, indicating exposure bias under fixed-length chunk training.

Figure 7

Figure 7: Beyond 113 frames, artifacts and reduced coherence emerge, demarcating the method's generalization limits under current training protocols.

Per-Category Analysis and the Role of Target Inversion

Motion category analysis reveals that TI offers maximum benefit in pan/tilt and fast/zoom scenarios—cases where structured camera trajectories yield more predictable transitions—while benefit under static camera conditions is marginal, affirming the hypothesis that TI constrains flow estimation most effectively when target latents are highly structured.

Figure 8

Figure 8: TI offers pronounced FID improvements in pan/tilt and zoom scenarios, negligible in static camera settings.

Figure 9

Figure 9: FVD reductions from TI are most evident in complex camera motion categories.

Theoretical and Practical Implications

The empirical evidence substantiates FlowC2S as a viable path for efficient, fast, and memory-conscious video continuation, substantiated by a design that forgoes auxiliary noise and instead utilizes natural temporal structure for efficient flow learning. This paradigm could inform methods for latent sequence modeling well beyond video, motivating similar strategies in spatiotemporal generative modeling. The use of inherent optimal couplings implements a practical, structure-aware proxy for tractable optimal transport in high-dimensional generative contexts.

The target inversion mechanism bridges the objective shift encountered in non-Gaussian flow mappings and provides a simple template for leveraging target-side information for improved fine-tuning. In practical application, FlowC2S's reduction in memory and compute cost has direct relevance for real-time content generation in edge/AR/VR deployments.

Limitations and Future Directions

FlowC2S, as currently framed, is limited by its exposure during training to fixed-length, transition-sanitized datasets. Future trajectories include:

  • Incorporating heterogeneous chunk lengths and more complex scene dynamics for enhanced generalization and consistency during arbitrarily long continuations.
  • Integrating conditioning signals for controllability—text prompts, motion fields, scene context—enabling user-directed, structured, or style-preserving video continuation, in line with the drive toward conditional controllable generative frameworks.
  • Rethinking how sequence length, temporal dependencies, and inter-chunk transitions are modeled to further close the gap between short-horizon learning and long-range stability.

Conclusion

FlowC2S demonstrates that direct modeling of the flow between observed and future video frames, with structure-aware coupling and target-augmented estimation, achieves competitive or superior results to leading video continuation methods under dramatically improved efficiency. It advances the practical utility of autoregressive video generation by making both inference and deployment significantly more tractable while maintaining high visual and temporal fidelity (2604.17625).

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Tweets

Sign up for free to view the 2 tweets with 0 likes about this paper.