MegaFlow: Zero-Shot Large Displacement Optical Flow

Abstract: Accurate estimation of large displacement optical flow remains a critical challenge. Existing methods typically rely on iterative local search or/and domain-specific fine-tuning, which severely limits their performance in large displacement and zero-shot generalization scenarios. To overcome this, we introduce MegaFlow, a simple yet powerful model for zero-shot large displacement optical flow. Rather than relying on highly complex, task-specific architectural designs, MegaFlow adapts powerful pre-trained vision priors to produce temporally consistent motion fields. In particular, we formulate flow estimation as a global matching problem by leveraging pre-trained global Vision Transformer features, which naturally capture large displacements. This is followed by a few lightweight iterative refinements to further improve the sub-pixel accuracy. Extensive experiments demonstrate that MegaFlow achieves state-of-the-art zero-shot performance across multiple optical flow benchmarks. Moreover, our model also delivers highly competitive zero-shot performance on long-range point tracking benchmarks, demonstrating its robust transferability and suggesting a unified paradigm for generalizable motion estimation. Our project page is at: https://kristen-z.github.io/projects/megaflow.

• The paper presents a unified zero-shot model for optical flow and point tracking by combining pretrained Vision Transformers with CNN fusion.
• It employs global all-pairs matching and a recurrent refinement module to ensure sub-pixel accuracy and strong temporal consistency.
• MegaFlow achieves state-of-the-art performance on benchmarks like Sintel and KITTI, reducing errors notably for extreme large displacements.

MegaFlow: Zero-Shot Large Displacement Optical Flow

Introduction and Motivation

MegaFlow proposes a new paradigm for optical flow estimation with a particular emphasis on handling large displacement scenarios and enabling zero-shot cross-domain generalization. Traditional optical flow algorithms, including classical optimization-based formulations and deep learning models such as RAFT, PWC-Net, and FlowFormer, often rely on local search or domain-specific fine-tuning. This architectural bias severely restricts robustness to extreme displacements and out-of-distribution motions. MegaFlow addresses these limitations by leveraging global matching mechanisms rooted in pretrained Vision Transformers, specifically DINOv2 and VGGT, and coupling them with lightweight local refinement modules to achieve temporally consistent and high-fidelity motion fields.

Architecture Overview

MegaFlow's architecture consists of three primary components. First, a feature extraction stage employs a frozen DINOv2 Vision Transformer and a trainable ResNet-based CNN to encode both global semantic context and local structural details. Features from multiple frames undergo fusion via alternating frame-wise and global-attention transformer blocks, producing globally consistent representations suitable for dynamic motion estimation.

Second, a global matching process establishes initial correspondences by computing dense all-pairs feature correlations between frames, followed by expectation-based coordinate assignment. This enables MegaFlow to robustly resolve large spatial displacements without the ambiguities inherent in local search.

Third, a recurrent refinement module iteratively improves the initial flow estimates. This module integrates spatial convolutions and temporal attention, ensuring sub-pixel accuracy and temporal consistency. Notably, the architecture natively supports variable-length inputs, allowing seamless adaptation to multi-frame motion estimation tasks.

Figure 1: The pipeline of MegaFlow combining DINOv2 and CNN features, global attention and matching, followed by recurrent local refinement for variable-length sequences.

Core Methodology

The feature extraction process fuses patch tokens from DINOv2 with multi-scale CNN features using a DPT-style head and pixel unshuffle. The global matching computes an all-pairs correlation volume, applying softmax normalization to produce correspondence distributions and initial flows. Subsequent recurrent refinement upsamples flows to CNN resolution and constructs local correlation volumes for iterative updates via ConvNeXt and temporal attention blocks.

Training objectives include a smooth $\ell_1$ loss on both initial and refined flows, with exponentially-weighted supervision across refinement iterations. The model's tracking extension computes displacement fields between the query frame and arbitrary target frames using global matching, enabling unified dense point tracking and zero-shot adaptation to long-range tasks.

Zero-Shot Performance and Generalization

MegaFlow demonstrates distinctive state-of-the-art performance in zero-shot settings on multiple public optical flow and point tracking datasets. On the Sintel benchmark, the model achieves unprecedented EPE values—0.89 for Sintel (Clean) and 1.83 for Sintel (Final). It delivers robust scores on KITTI (Fl-all of 10.7). These outcomes highlight enhanced resilience to appearance changes, motion blur, and occlusion, outperforming both task-specific baselines and models trained on more diverse correspondence datasets.

MegaFlow substantially outperforms iterative local methods on extreme large displacements ($s_{40+}$), with EPE dropping to 4.729 on Sintel (Clean) and 11.175 on Sintel (Final), flattening the error curve as displacement increases.

Figure 2: MegaFlow surpasses prior optical flow models on Sintel and TAP-Vid, maintaining low error even for large displacements and long-range point tracking.

Unified Point Tracking and Cross-Task Transfer

MegaFlow's architecture inherently supports dense point tracking without modification. Zero-shot evaluations on TAP-Vid yield average $\delta_{avg}$ scores of 73.6%, surpassing all prior flow-based baselines and rivaling specialized trackers (e.g., LocoTrack, BootsTAPIR, CoTracker3). Fine-tuned variants on Kubric achieve a new benchmark with 79.6% average accuracy, outperforming dedicated trackers even with more extensive supervision.

Qualitative visualizations illustrate MegaFlow's coherent and stable trajectories over long sequences (up to 600 frames), maintaining sharp local structures and temporal consistency regardless of initialization or occlusions.

Figure 3: MegaFlow produces temporally stable, precise tracks and flow estimates on prolonged sequences, outperforming local search methods in accuracy and consistency.

Benchmark Analysis and Ablations

MegaFlow retains high competitiveness across multiple benchmarks without fine-tuning. On Spring, it achieves lowest EPE and Fl-all, proving scalability to Full HD resolutions. On KITTI, the architecture retains comparable precision to multi-frame models and foundation-based approaches, despite zero-shot evaluation and fixed patch tokenization.

Ablation studies confirm the importance of pretrained vision priors (VGGT, DINOv2) and feature fusion. Scaling transformer depth without pretrained initialization consistently degrades performance, while freezing or omitting CNN fusion increases outlier rates. Multi-frame context with temporal attention drastically improves temporal consistency and boundary accuracy, especially on Sintel. The number of recurrent refinement iterations ($K$) is optimally set at eight, balancing accuracy and computational efficiency.

Figure 4: MegaFlow generates sharper motion boundaries and more precise flow fields compared to SEA-RAFT, MemFlow, and WAFT-DINOv3-a2 on challenging benchmarks.

Implications and Future Directions

MegaFlow demonstrates that global vision priors and global correspondence matching, complemented by iterative refinement, provide a scalable, generalizable foundation for dense motion estimation. The architecture's adaptability enables robust application to both optical flow and dense point tracking, unifying previously disparate domains. Practically, this offers substantial advantages in real-world scenarios characterized by domain shifts, extreme displacements, and long-range correspondence requirements.

Theoretically, MegaFlow's transferability suggests the emergence of foundation models for dense motion that parallel developments in image classification and semantic segmentation. Future work may focus on joint pretraining regimes integrating optical flow and tracking tasks, as well as architectural optimizations for reducing computational overhead in multi-frame modeling.

Conclusion

MegaFlow establishes a unified, scalable architecture for extreme optical flow and long-range point tracking, combining pretrained vision priors, global matching, and iterative local refinement. The model achieves state-of-the-art zero-shot performance across diverse benchmarks, robustly handles large displacements, and enables cross-task transfer without architectural modifications. MegaFlow lays groundwork for universal dense motion models, advancing both practical performance and theoretical frameworks in visual correspondence.