MA-V: Extensive Video Matting Dataset

• The paper presents a scalable pseudo-labeling pipeline using VideoMaMa to convert segmentation masks into pixel-accurate alpha mattes.
• The dataset includes over 50,000 diverse real-world videos, enhancing the robustness and generalizability of video matting models.
• Empirical results with SAM2-Matte demonstrate improved in-the-wild performance by leveraging generative diffusion priors in annotation.

The Matting Anything in Video (MA-V) dataset is a large-scale video matting resource constructed to address the scarcity of labeled data for generalizing video matting models to real-world scenarios. Developed via a scalable pseudo-labeling pipeline based on the Video Mask-to-Matte Model (VideoMaMa), MA-V provides high-quality matting annotations for over 50,000 real-world videos, encompassing a broad diversity of scenes and motion types. The dataset serves as a foundation for advancing the robustness and generalizability of video matting methods, underscoring the impact of leveraging generative priors and accessible segmentation guides in dataset creation (Lim et al., 20 Jan 2026).

1. Motivation and Context

Video matting requires precise pixel-level separation of foreground objects from the background across frames, producing temporally consistent alpha mattes. The primary challenge in progressing video matting research is the lack of large-scale, high-quality annotated data from in-the-wild sources. Prior models have exhibited limited generalization to real-world footage due to this data bottleneck. The introduction of the MA-V dataset directly addresses this limitation by enabling the training and evaluation of models on a far more diverse and representative collection of real-world video content (Lim et al., 20 Jan 2026).

2. Dataset Construction and Pseudo-Labeling Pipeline

The MA-V dataset is generated through a scalable pseudo-labeling pipeline. Central to this process is the VideoMaMa model, which converts coarse segmentation masks into pixel-accurate alpha mattes. VideoMaMa leverages pretrained video diffusion models to enhance mask-to-matte conversion performance, even in the absence of dense ground-truth mattes for real-world video. The pipeline applies this model across a wide selection of raw video sequences, utilizing accessible segmentation cues as mask input. The resulting pseudo-labels are automatically produced, enabling the creation of high-quality annotations at a scale infeasible with manual labeling (Lim et al., 20 Jan 2026). This suggests a significant advance in annotation efficiency and diversity relative to previous video matting datasets.

3. Scale, Coverage, and Diversity

MA-V comprises annotated mattes for more than 50,000 real-world videos. The data spans a wide array of scenes and motion patterns, providing diversity in content, backgrounds, object types, and movement dynamics. This scale and coverage position MA-V as a leading resource in video matting and as a basis for robust model training and evaluation. A plausible implication is that models trained on MA-V benefit from improved in-the-wild generalization due to the dataset's breadth.

4. Annotation Quality and Benchmarking

The pseudo-labels generated for MA-V annotation are produced by VideoMaMa, which demonstrates strong zero-shot generalization to real-world video, despite being trained solely on synthetic data. The dataset's quality is empirically validated by fine-tuning the SAM2 model on MA-V, resulting in the SAM2-Matte variant. SAM2-Matte yields improved robustness in in-the-wild video matting tasks compared to the same model trained on existing matting datasets (Lim et al., 20 Jan 2026). This outcome highlights MA-V's value for benchmarking and for advancing state-of-the-art in video matting models.

5. Practical Applications and Research Impact

The availability of MA-V enables several practical and research advances:

• Model Training: Facilitates training of deep video matting architectures that require large and diverse annotation resources.
• Robustness Evaluation: Provides a broad benchmark for assessing real-world generalization, particularly to challenging, previously unseen motion/scene combinations.
• Dataset-Driven Methodology: Demonstrates the effectiveness of scalable pseudo-labeling using generative matting priors as an engine for dataset creation.
• Transfer Learning Validation: Empirical findings, including SAM2-Matte performance, validate MA-V's utility for transfer learning and robustification of matting models (Lim et al., 20 Jan 2026).

6. Significance for Future Research

The construction and validation of MA-V exemplify a data-centric approach to video matting, emphasizing the centrality of scalable, high-diversity pseudo-labeled resources in overcoming generalization bottlenecks. The dataset's scale and annotation quality enable new research into generalizable matting algorithms and promote cross-benchmark comparisons. MA-V also illustrates the broader potential of generative diffusion-prior models and segmentation-guided pseudo-labeling in constructing datasets for dense vision tasks in unconstrained real-world scenarios (Lim et al., 20 Jan 2026).