MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

Abstract: We propose MotionAtlas, a system for detailed captioning of motion-centric videos, comprising (1) a dedicated human-annotated benchmark, (2) a scalable, high-quality pipeline to construct training samples, and (3) a family of powerful Video-MLLMs. Unlike conventional global motion captioning datasets, we focus on region-aware motion captioning: given a video and a spatiotemporal mask, the model generates precise descriptions of motion within the target region, thereby alleviating visual clutter and motion entanglement and enabling reliable, quantifiable evaluation. Concretely, we first build MotionAtlas-Bench, a comprehensive benchmark comprising 2,073 multiple-choice questions, meticulously annotated for a curated set of high-quality, motion-centric videos, to evaluate fine-grained motion understanding of the objects in question. Second, we design a rigorous and scalable data pipeline that leverages self-bootstrap refinement to suppress fine-grained hallucinations, yielding 159k high-quality motion captioning data. Third, we design a tailored training data composition strategy, which achieves consistent and substantial performance gains across diverse baseline Video-MLLMs, including Molmo2 and Qwen3-VL. For instance, MotionAtlas-4B surpasses Qwen3-VL-4B by an average of 5.2 percentage points across general motion benchmarks. The benchmark, dataset, and code have been released.

• The paper demonstrates a novel region-level motion captioning framework that disentangles local motion dynamics from global scene context.
• It presents a scalable annotation pipeline with automated segmentation, dual captioning, and self-bootstrap refinement, producing 159k detailed captions.
• Empirical benchmarks reveal significant accuracy gains, underscoring the need for precise spatial and temporal grounding in video understanding.

MotionAtlas: Region-Level Motion Captioning for Fine-Grained Video Understanding

Motivation and Problem Formulation

Numerous advancements in Video-MLLMs have resulted in strong performance on general video understanding, yet fine-grained region-level motion reasoning remains an unsolved challenge. Existing datasets and benchmarks enforce global scene captioning, which entangles visual context, leads to evaluation intractability, and fails to reliably incentivize models to capture local, complex dynamics. The global paradigm is highly susceptible to both recall drop and hallucinated contents, due to the inability to verify spatial or temporal completeness.

MotionAtlas addresses these limitations by formally shifting to region-aware motion captioning. Given a video and a spatiotemporal mask, the model must precisely describe only the localized motion within the target region, inherently disentangling local events and enabling scalable, dense, and diagnostic supervision—supported by MCQ-based checklist evaluation. This targeted setting allows for quantifiable ablation of model, data, and annotation pipeline, providing a robust framework to drive future advances in video grounding and action understanding.

MotionAtlas-Bench: Benchmark Design and Structure

MotionAtlas-Bench is constructed to operationalize region-level motion evaluation through an event–motion–fact annotation hierarchy. Each video is segmented into events; each event is densely described and decomposed into atomic, verifiable facts, forming a set of MCQs for diagnostic evaluation.

Figure 1: Illustration of the MotionAtlas-Bench pipeline, from video decomposition into events with spatial masks, to fact extraction and MCQ checklist formation for precise evaluation.

MotionAtlas-Bench features:

• Dense coverage: 2,073 MCQs across 107 videos, with an average of 19.4 MCQs per video, spanning key aspects—Spatial, Parts, Kinematics, Interaction, State, and Camera.
• Granularness: Target entities are small (mean area 14% of frame), with high motion complexity, directly targeting failure modes in existing models (object tracking, small part analysis, spatiotemporal entanglement).
• Diagnostic objectivity: Each MCQ specifically targets an atomic attribute, allowing for fine-grained recall, accuracy, and precision metrics.

An extensive human–AI collaboration protocol ensures annotation quality: VLMs propose events/facts, humans refine both event boundaries and factual claims, and a two-tiered verification process eliminates low-discriminativity or hallucinated MCQs (shown in Figure 2 and Figure 3).

Figure 2: Human refinement protocol for event segmentation, merging/correcting boundaries for consistent temporal semantics.

Figure 3: Guidelines for MCQ factual verification—ensuring option discriminability and factual grounding.

Distribution analyses confirm comprehensive coverage (see Figure 4 and Figure 5).

Figure 4: MotionAtlas-Bench aspect distribution, emphasizing fine body articulation, spatial relationship, and local kinematics.

Figure 5: Histogram of events per sample, demonstrating multi-event complexity per video.

Qualitative benchmark cases further illustrate typical model failure modes: omission of subtle local motion cues and incorrect attribute assignments (Figure 6).

Figure 6: Benchmark examples with missed and incorrect motion attributes highlighted, indicating persistent challenges in current models.

Data Pipeline: Scalable, High-Fidelity Region Motion Captioning

Fine-grained region-level annotation at scale is inhibited by the cost of human review and persistent VLM hallucinations in long-form event descriptions—especially at event boundaries and in reasoning over small, mobile entities. MotionAtlas introduces a robust, highly scalable annotation pipeline consisting of:

1. Segment: Automated event segmentation.
2. Caption: Local (event-cropped) and global (full-video) captioning.
3. Self-bootstrap refinement: Dual VLM rollouts identify claim divergences, which are re-verified by LLM assessment of differences, suppressing hallucinations and enforcing factual consistency.
4. Multi-source narrative synthesis: Local and global narratives are merged to yield temporally coherent, non-redundant, high-dense descriptions.

Ablations show that each pipeline stage is essential: self-bootstrap eliminates hallucinated content ($-3.5\%$ accuracy if omitted), full-video captioning ensures temporally consistent synthesis ($-6.7\%$), and spatial crops enhance entity salience ($-7.2\%$).

This pipeline produces MotionAtlas-Data: 159k high-fidelity region event captions, averaging 23 verbs and 212 words per sample, with unmatched verb and motion density among all motion-oriented video datasets.

Model Training and Data Integration

To maximize transfer to general video reasoning, the authors compose an SFT mixture of region-aware captioning (MotionAtlas-Data), general motion captioning, motion QA, and vision–language QA. Sampling is proportional to dataset size, enabling robust generalization and mitigating catastrophic forgetting.

Quantitative and Qualitative Results

On MotionAtlas-Bench, all open and closed models perform poorly under region-level MCQ evaluation—e.g., Gemini 3 Pro, GPT-5.2, and Qwen3-VL-235B are all below 38% overall accuracy. Fine-tuning with MotionAtlas-Data yields pronounced gains: up to +8.4 points for Qwen3-VL-4B, and considerable improvements even for larger models. Single-frame vs. full-sequence grounding ablates tracking accuracy vs. pure motion understanding, with clear boosts in the full-sequence protocol.

Figure 7: Consistent improvement of MotionAtlas-over baseline models (Qwen3-VL), underlining the impact of region-level data.

Externally, on standard video benchmarks (e.g., MotionBench, DREAM-1K, TEMP-COMPASS, FAVOR-Bench), MotionAtlas-trained models achieve improvements up to +7.8 (TOMATO) and +8.1 (FAVOR-Bench), confirming effective transfer from region-level supervision. Ablations prove that these gains cannot be attributed only to more verbs or increased sample size—region detail and explicit spatial grounding are necessary.

Figure 8: Data scaling curves: increasing MotionAtlas-Data scale produces monotonic, non-saturating performance improvements on all benchmarks.

Qualitative comparisons show that prior datasets drive coarse motion description, whereas MotionAtlas yields detailed, temporally explicit, spatially-grounded articulation of dynamic actions (Figure 9 and Figure 10).

Figure 9: Qualitative comparison of baseline (coarse) and MotionAtlas (dense, detailed) event captions.

Figure 10: Examples of MotionAtlas captions, exhibiting rich, multi-aspect motion narratives.

Implications and Future Directions

On the practical side, MotionAtlas-Data and the MCQ-bench paradigm provide a highly diagnostic testbed for Video-MLLMs, precisely localizing model errors to specific subregions, action parts, or temporal boundaries. This opens the avenue for multi-granular evaluation and event localization, critical for embodied AI, robust video QA, and downstream robotics/Autonomous Driving.

Theoretically, the decompositional pipeline validates that region-level supervision robustly transfers to global video understanding, as the core model challenge lies not in scene parsing but in spatiotemporal subsampling and grounding—limitations endemic to current global/weakly supervised training paradigms. Future research can exploit region-entity interaction modeling, support multi-referent scenarios, and study the compositional spatiotemporal alignment at the event and object-network level.

The MotionAtlas approach can further be extended with explicit evidence chains for explainability, causal reasoning over grounded regions, and comprehensive joint analysis with appearance and affordance cues.

Conclusion

MotionAtlas delivers a scalable, highly structured framework for region-level motion captioning and evaluation, with an accompanying data pipeline and benchmark that directly address persistent failure modes in current Video-LLMs. Empirical results substantiate the necessity of region-aware supervision for both fine-grained and general motion understanding. The MCQ-based protocol and data methodology are poised to underpin the next generation of spatiotemporally robust Video-MLLMs for a wide spectrum of applications.