Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (2512.08765v1)

Abstract: We present Wan-Move, a simple and scalable framework that brings motion control to video generative models. Existing motion-controllable methods typically suffer from coarse control granularity and limited scalability, leaving their outputs insufficient for practical use. We narrow this gap by achieving precise and high-quality motion control. Our core idea is to directly make the original condition features motion-aware for guiding video synthesis. To this end, we first represent object motions with dense point trajectories, allowing fine-grained control over the scene. We then project these trajectories into latent space and propagate the first frame's features along each trajectory, producing an aligned spatiotemporal feature map that tells how each scene element should move. This feature map serves as the updated latent condition, which is naturally integrated into the off-the-shelf image-to-video model, e.g., Wan-I2V-14B, as motion guidance without any architecture change. It removes the need for auxiliary motion encoders and makes fine-tuning base models easily scalable. Through scaled training, Wan-Move generates 5-second, 480p videos whose motion controllability rivals Kling 1.5 Pro's commercial Motion Brush, as indicated by user studies. To support comprehensive evaluation, we further design MoveBench, a rigorously curated benchmark featuring diverse content categories and hybrid-verified annotations. It is distinguished by larger data volume, longer video durations, and high-quality motion annotations. Extensive experiments on MoveBench and the public dataset consistently show Wan-Move's superior motion quality. Code, models, and benchmark data are made publicly available.

• The paper introduces a latent trajectory guidance scheme that injects motion signals by propagating first-frame latent features, eliminating the need for extra modules.
• It demonstrates superior performance with metrics like FID (12.2) and FVD (83.5) on MoveBench compared to state-of-the-art baselines.
• The approach supports diverse motion control applications, including object/camera motion, motion transfer, and video editing with low inference latency.

Motion-Controllable Video Generation with Latent Trajectory Guidance: An Expert Summary of "Wan-Move"

Introduction and Motivation

"Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance" (2512.08765) proposes a scalable framework for precise, high-fidelity, and flexible motion-controlled video generation. The authors address the core challenge in image-to-video (I2V) synthesis: how to inject explicit, fine-grained motion guidance into large video diffusion backbones without cumbersome and scale-limiting architectural changes. Existing methods predominantly rely on auxiliary modules—such as motion encoders or fusion networks—to represent and integrate motion signals, resulting in signal degradation, increased inference latency, and limited scalability.

Wan-Move instead advocates injecting motion directly into the condition features of a pre-trained I2V backbone by propagating first-frame latent features along dense, spatially-mapped point trajectories. This eliminates the need for extra modules, enabling efficient and precise control over object and camera motion, robust scalability, and fast convergence during fine-tuning.

Figure 1: Wan-Move is an image-to-video generation framework supporting diverse motion control applications, producing high-fidelity, motion-accurate samples (832×480p, 5s).

Methodology

Latent Trajectory Guidance

At the core of Wan-Move is a latent trajectory guidance scheme. The process involves encoding dense or sparse point trajectories—representing desired object or scene motions—from pixel space to the video model's latent space. The original first-frame latent feature map is then updated by replicating features along these trajectories across subsequent frames. This propagates appearance and contextual feature information, allowing the model to animate scene elements according to precise trajectories.

Figure 2: (a) Motion guidance injection by transferring point trajectories to latent space and propagating first-frame features along each trajectory. (b) Wan-Move is trained using an existing image-to-video model, with an efficient latent feature replication step to update the condition feature.

This approach leverages the translation-equivariant property of the latent space learned by the underlying VAE, ensuring local consistency and rich semantic transfer. No additional motion encoder or fusion network is introduced—motion information is injected solely by modifying the condition features.

Training and Inference

Wan-Move is initialized from a strong pre-trained I2V model (e.g., Wan-I2V-14B). Training utilizes a large-scale, high-quality video dataset filtered for visual and motion consistency. Trajectories are tracked using CoTracker across a dense spatial grid, with a mixed sampling strategy for the number of tracks to enhance robustness. The objective utilizes flow matching loss to learn to transform noise into data samples conditioned on motion-augmented features.

During inference, user-specified (or system-extracted) trajectories are lifted into latent space, and latent feature replication guides synthesis. Classifier-free guidance is employed for conditional fidelity.

MoveBench: Benchmark Design and Rationale

The lack of comprehensive, high-quality benchmarks for motion-controllable video synthesis motivates MoveBench. MoveBench contains 1,018 curated videos, each 5 seconds long at 480×832 resolution, with both dense trajectory and mask annotations across 54 content categories. A four-stage curation pipeline ensures coverage and diversity, while hybrid human+SAM labeling provides precise motion annotations for single and multi-object scenarios.

Figure 3: MoveBench construction pipeline for high-quality, richly annotated samples.

Figure 4: Balanced sample number per class confirms diverse content stratification.

MoveBench provides a rigorous testbed for evaluating both pixel-level video fidelity and trajectory-level motion guidance accuracy, with captions for multi-modal conditioning.

Figure 5: The interactive annotation interface for motion trajectory specification, leveraging SAM for rapid mask generation and user review.

Experimental Results

Wan-Move is evaluated against state-of-the-art academic and commercial baselines, including Tora, MagicMotion, LeviTor, ImageConductor, and Kling 1.5 Pro, on MoveBench and DAVIS.

Numerical Results

Wan-Move achieves the strongest numerical performance across standard quality metrics—FID, FVD, PSNR, SSIM, as well as EPE (end-point error) for motion accuracy. On MoveBench, Wan-Move achieves FID of 12.2, FVD of 83.5, PSNR of 17.8, SSIM of 0.64, and EPE of 2.6—outperforming all baselines. This superiority is consistent in multi-object motion and stress tests on large-motion and out-of-distribution scenarios, where the performance gap over competing methods widens.

Qualitative Comparisons

Visual comparisons reveal Wan-Move yields higher fidelity and motion precision; failure cases in competing methods include deviation from specified trajectories and significant visual artifacts.

Figure 6: Qualitative comparisons between Wan-Move and recent approaches, highlighting deviations (red markers) in motion fidelity in prior methods.

Figure 7: Additional qualitative comparisons with MagicMotion under sparse input (bounding box) conditions.

Figure 8: Wan-Move demonstrates better motion and visual quality relative to Tora.

Human Studies

In two-alternative forced choice tests, Wan-Move is preferred (>96% win rates) against all research solutions for motion accuracy, quality, and fidelity. Notably, it matches or exceeds commercial Kling 1.5 Pro in overall visual quality and narrows the gap significantly in motion metrics.

Ablations

Ablation studies substantiate the advantage of latent feature replication over pixel replication and random embedding: only the proposed approach achieves competitive EPE and image quality. Furthermore, the direct concatenation strategy is shown to be more efficient than heavy ControlNet-based fusion, yielding similar or better motion control with significantly reduced inference latency. Wan-Move generalizes to denser trajectories than seen in training, and its fidelity remains robust when no motion control is provided (default I2V setting).

Applications and Capabilities

Wan-Move supports a wide array of motion control paradigms:

• Object motion control via single/multi-point trajectories
• Camera control by manipulating trajectory clouds or following estimated camera paths

  Figure 9: Flexible camera control examples with synthesized linear, dolly-in, and dolly-out motions.

• Motion transfer by copying dense trajectories from a source video to synthesize new content

  Figure 10: Near-perfect video motion copy with 1,024 dense trajectories—appearance and object-level motion alignment closely matches the original.

• Video editing by coupling with image editing models for modified appearance while preserving motion

  Figure 11: Video editing via motion copy after first-frame modification using image editing models.

• 3D object rotation by estimating depth, applying 3D transformations, and projecting updated trajectories to the image plane

  Figure 12: Synthesis of 3D object rotation controlled by trajectory transformations in image space.

Analysis of Limitations

While Wan-Move generalizes well, performance degrades for long-term or permanently missing trajectory segments due to occlusion—loss of motion signal leads to uncontrolled motion in occluded intervals. It also faces challenges in visually complex or crowded scenes and may occasionally synthesize motion violating physical plausibility, often traceable to erroneous tracking.

Figure 13: Representative failure modes—motion control loss on persistent trajectory disappearance, generation artifacts in complex scenes, and synthesized motion violating physical constraints.

Implications and Future Directions

Wan-Move demonstrates that precise, scalable motion control can be realized via trajectory-guided feature injection in latent space, removing the need for dedicated architectural changes and auxiliary encoders. This has notable implications for both practical deployment (due to low overhead and compatibility with scaling) and theoretical work (suggesting the sufficiency of translation-equivariant latent spaces for feature propagation along arbitrary trajectories).

With the release of MoveBench as a benchmark, the field is equipped to perform rigorous, large-scale, and granular evaluation of motion-controllable generative models.

There is potential for further innovations in:

• Integration of more sophisticated trajectory estimation and editing tools
• Granular control over intrinsic/extrinsic motion factors (e.g., disentangling camera and object motion)
• Robustness under severe occlusion
• Broader application domains including simulation, robotics, and interactive design environments.

Conclusion

"Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance" (2512.08765) introduces a streamlined and effective method for injecting precise motion signals into large-scale I2V models, validated by both strong empirical and human paper results. Its latent trajectory guidance scheme is computationally efficient, flexible across motion types, and robust against existing baselines. The approach substantially advances the state of controllable video generation both practically and as a foundation for future research directions.