EcoVideo: Entropy-Orchestrated Video Generation Paradigm in Cloud-Edge Dynamics

Abstract: DiT video generation is latency-intensive due to iterative full-frame denoising, while prior cloud-edge methods largely rely on static inter-step decoupling and cannot leverage inter-frame similarity or adapt to system dynamics. We propose EcoVideo, an entropy-orchestrated framework for dynamic inter-frame decoupling: early-stage self-attention entropy provides a training-free estimate of frame-wise information density for frame selection; a cloud large model denoises sparse high-entropy keyframes; and an edge lightweight model reconstructs the remaining frames via motion-aware interpolation with refinement for temporal stability. EcoVideo further adapts the keyframe budget and edge refinement depth to real-time bandwidth and compute availability, optimizing end-to-end latency under constraints. Experiments on representative DiT video generators show improved quality--efficiency trade-offs and up to 2.9x end-to-end speedup in low-bandwidth, compute-limited edge settings. Code is available at https://github.com/IF-LAB-PKU/EcoVideo.

• The paper introduces an entropy-oriented paradigm that dynamically selects keyframes for cloud denoising, enabling efficient video generation under fluctuating resources.
• It employs a training-free attention entropy analysis and motion-aware EcoVFI interpolation to reconstruct intermediate frames, reducing redundant computations and latency.
• Experimental results show up to 2.9× speedup and dramatic reductions in communication volume, achieving improved temporal consistency and high-fidelity outputs.

EcoVideo: Entropy-Orchestrated Cloud-Edge Video Generation

Introduction and Motivation

Diffusion Transformer (DiT)-based video generation has established itself as a high-fidelity paradigm for spatiotemporal synthesis but is intrinsically constrained by the stepwise denoising of full-frame sequences, yielding prohibitive latency and computation costs. Existing cloud-edge collaborative frameworks, such as HybridSD and EC-Diff, attempt to alleviate these overheads via inter-step decoupling, partitioning inference timesteps between cloud and edge. However, these approaches fail to exploit temporal redundancy and are unable to adapt to fluctuating transmission and computation resources, frequently resulting in suboptimal end-to-end latency and degraded video fidelity due to artifacts like texture flickering and detail collapse.

EcoVideo (2606.30557) proposes a fundamentally different approach: reframing cloud-edge collaboration as a dynamic, entropy-driven, inter-frame decoupling paradigm. It introduces a training-free attention-entropy analysis to select keyframes for high-capacity cloud denoising, with a lightweight edge module reconstructing intermediate frames via motion-aware interpolation. Critically, EcoVideo incorporates a resource-aware adaptation mechanism, dynamically rebalancing keyframe selection and edge refinement according to real-time bandwidth and computational budgets. This architecture achieves substantially lower latency, robust adaptation to environmental shifts, and improved temporal coherence.

Paradigm Shift: From Inter-Step to Inter-Frame Decoupling

Conventional stepwise decoupling strategies rigidly allocate early denoising steps to the cloud and late steps to the edge, treating all frames as equally salient. This results in repeated, redundant computation across highly similar frames and entrusts the edge with complex denoising tasks for which it is ill-suited—causing fidelity degradation, particularly in dynamic regions.

EcoVideo’s key innovation is to quantify the information density of each frame using attention entropy computed during early denoising steps. Frames with high entropy—corresponding to periods of complex motion, structural transitions, or significant content—are earmarked as keyframes for cloud-side high-fidelity denoising. Intermediate, low-entropy frames, which are likely to be structurally and semantically redundant, are reconstructed on the edge via dense conditioning on keyframes. This localized workload shifting suppresses redundant cloud computation and transmission, providing a structurally grounded foundation for efficient collaborative inference.

Figure 1: A comparison between inter-step decoupling (HybridSD) and inter-frame decoupling paradigms (EcoVideo).

Framework Overview and Technical Contributions

EcoVideo consists of three core modules:

1. Frame-wise Attention Entropy Analysis:

Attention weights are leveraged to compute token-level entropies, aggregated per frame and smoothed via EMA during an initial warm-up. High-entropy frames are adaptively prioritized as keyframes, establishing a sparsity-aware optimization boundary.

Figure 2: Overview of EcoVideo, illustrating entropy estimation, keyframe selection, and cloud-edge orchestration.

2. Entropy-Orchestrated Generation:

Keyframe spans anchor the global motion and appearance, denoised exclusively by the cloud model. Non-keyframe latents are injected as contextual, stop-gradient tokens to maintain global structure awareness. The edge-side uses EcoVFI—a modified, difficulty-aware interpolation framework utilizing density, motion, structure, and texture cues—to guide midframe reconstruction and dynamic budget allocation for challenging intervals, minimizing temporal discontinuities.

Figure 3: Frame-level entropy-orchestrated generation with model collaboration on keyframes and motion/structure-conditioned edge interpolation on intermediate frames.

3. Bandwidth- and Compute-Adaptive Configuration: A lightweight scheduler continually profiles bandwidth and compute resource availabilities, performing exhaustive search over keyframe density and interpolation depth to maximize a composite utility function balancing quality and latency. EMA-updated unit costs for cloud processing, transmission, and interpolation allow accurate, low-overhead configuration updates on each scheduling cycle.

Experimental Analysis

EcoVideo is evaluated on state-of-the-art DiT backbones, including Wan2.1, Wan2.2, and CogVideoX models, across varied system resource regimes.

Quantitative Results:

EcoVideo achieves up to 2.9× end-to-end speedup in bandwidth- and compute-constrained edge settings, outperforming HybridSD and EC-Diff baselines on both VBench [zheng2025vbench] quality metrics and system throughput. Notably, EcoVideo reduces communication volume by an order of magnitude (e.g., 1.10 MB vs. 17.23 MB on Wan2.1-14B) and lowers edge latency (e.g., 96.23 s vs. 648.92 s for HybridSD) while improving perceptual and structural consistency.

Qualitative Comparison:

Qualitative results demonstrate substantial reductions in temporal artifacts. Unlike stepwise split baselines that exhibit detail loss and ghosting in dynamic transitions, EcoVideo preserves structural continuity and motion realism, especially in sequences with high inter-frame entropy.

Figure 4: Qualitative comparison on Wan2.2 and CogVideoX; EcoVideo mitigates flicker and collapse, maintaining high-fidelity temporal coherence.

Ablation Studies:

Ablations confirm the necessity of both entropy-driven keyframe selection and the EcoVFI interpolation. Uniform frame selection, interpolation-only strategies, and reduced entropy warm-up yield measurable drops in VBench scores, particularly in dynamic scenarios. The system-aware adaptation module demonstrates resilience to bandwidth reductions and compute contention, maintaining speedups where fixed-split baselines degrade or become slower than cloud-only inference.

Figure 5: Ablation and resource breakdown—EcoVideo's robustness under contention and bandwidth fluctuations is evident compared to baselines.

Keyframe Visualization:

Visualization of keyframes and interpolated frames further highlights the adaptive sparsity and fidelity trade-offs achieved by entropy orchestration.

Figure 6: Keyframe (green) and interpolated (yellow) frames, illustrating EcoVideo's dynamic selection and interpolation strategy.

Implications and Future Directions

EcoVideo denotes a significant advancement in efficient video generation under cloud-edge paradigms, coupling information-theoretic sparsity estimation with practical system adaptation. Its resource-aware, frame-level decoupling offers both theoretical generalization—by leveraging entropy as a proxy for content importance—and practical robustness—by maintaining quality under dynamically fluctuating real-world system constraints.

Potential avenues for further research include:

• Integration with cloud-side parallelism, caching, and sparse attention schemes for further acceleration of keyframe synthesis
• Adaptive uncertainty-aware keyframe insertion when encountering abrupt scene changes, fast motion, or heavy occlusion
• Jointly optimizing interpolation modules for cross-modal and longer horizon dynamics
• Extensive deployment and tracing on heterogeneous, real-world network infrastructures for long-term adaptation

Conclusion

EcoVideo establishes an entropy-orchestrated cloud-edge framework that shifts video generation collaboration from stepwise to frame-level granularity. Through dynamic entropy estimation, adaptive keyframe allocation, and robust, information-aware interpolation, the framework delivers significant reductions in end-to-end latency and communication overhead, while improving temporal consistency and visual quality over traditional approaches. Its systematic incorporation of runtime profiling and optimization renders it inherently adaptable to diverse deployment conditions and positions entropy-driven orchestration as a scalable approach for next-generation efficient video synthesis.