MoCentric-Bench: Motion-Centric Video Grounding

• MoCentric-Bench is a motion-centric benchmark for referring video segmentation that forces models to rely on genuine temporal cues.
• It employs innovative probes like static keyframe repetition and video reversal to differentiate motion understanding from static appearance reliance.
• Empirical results demonstrate that strong single-frame baselines degrade in split-screen settings, underscoring the need for true motion-aware models.

Searching arXiv for the MoCentric-Bench source paper and closely related benchmark context. MoCentric-Bench is a motion-centric benchmark for pixel-level visual grounding in videos, introduced in “PixFoundation 2.0: Do Video Multi-Modal LLMs Use Motion in Visual Grounding?” to test whether video multi-modal LLMs actually use motion when grounding language to objects, or whether they rely primarily on static appearance cues from one or a few frames (Siam, 2 Sep 2025). It is framed around referring video segmentation: given a video and a natural-language referring expression, the model must output the target object’s pixel mask over time. The benchmark is motivated by the claim that existing video grounding benchmarks, including motion-referring segmentation settings, often do not truly require motion understanding, because a single frame can frequently suffice via static evidence such as object category, color, position, heading, or scene context (Siam, 2 Sep 2025).

1. Problem formulation and motivation

The benchmark addresses the question posed by the underlying paper: “Do video MLLMs use motion in visual grounding?” The paper’s answer is largely negative: current methods can often be fooled by static cues, and their performance drops sharply when evaluation truly requires motion sensitivity (Siam, 2 Sep 2025).

The task is described verbally as referring video segmentation. Given a video and a referring expression, the system must segment the referred object or objects across the video. The paper does not provide a symbolic task equation, optimization objective, or explicit training-loss formula. The notation used in evaluation is metric notation: region similarity $\mathcal{J}$, contour accuracy $\mathcal{F}$, and their average, written in the extracted text as $\mathcal{J{paper\_content}F}$, which the source describes as the standard average of $\mathcal{J}$ and $\mathcal{F}$ (Siam, 2 Sep 2025).

A central premise is that many motion-referring expressions are not sufficiently motion-dependent in practice. The paper gives examples such as “jump to the left then jump back,” “cow shaking head and looking at us,” and “The little cat walking from behind to the front,” and argues that these can often be grounded from a single frame because one frame may reveal direction or heading, relative position, object category, and interaction context (Siam, 2 Sep 2025). This suggests that prior evaluation settings can overestimate genuine temporal reasoning ability.

2. Benchmark design and source data

MoCentric-Bench is synthesized primarily from MeVIS, especially its val_u subset and train subset, rather than being a wholly new raw-video collection (Siam, 2 Sep 2025). For evaluation, the paper uses MeVIS val_u, described as 50 videos with public ground truth, and MeVIS val, described as 140 videos evaluated via server. RefDAVIS17 validation is used in broader experiments, but MoCentric-Bench itself centers on the motion-centric variants of MeVIS val_u (Siam, 2 Sep 2025).

The benchmark is built around two major conceptual probes. The first is Motion Existence, which asks whether a model can distinguish true motion in the original video from fake motion represented by a repeated static frame. The second is Motion Order, which asks whether a model can distinguish original motion from the reverse of the same motion, paired with a reversed referring expression (Siam, 2 Sep 2025). These probes produce four concrete evaluation variants:

1. Single frame
2. Reverse video
3. Multi-video layout with single frame
4. Multi-video layout with reverse video (Siam, 2 Sep 2025)

The split-screen multi-video layouts are especially important because they transform the task from simply segmenting an object to segmenting the object in the stream whose motion matches the language (Siam, 2 Sep 2025). The paper presents this as the core contribution of MoCentric-Bench.

The reported benchmark statistics are specific to the synthesized motion-centric variants. For the reverse variants, the paper reports 32 videos and 152 segmentation-motion expression pairs. For the single-frame variants, it reports 50 videos and 793 segmentation-motion expression pairs. The reverse set is smaller because some expressions could not be reversed in a way that was sufficiently distinguishable from the original and were manually removed (Siam, 2 Sep 2025).

3. Motion-centric probing techniques

The Single frame probe tests whether a model can be fooled by fake motion. The authors automatically select a keyframe likely to best capture the motion expression. They use Qwen2.5-VL as a coarse temporal grounding tool with the prompt: “Given the query: <EXP>, when does the described content occur in the video? Output the first and last seconds for this action in JSON format.” They then identify the temporal window and choose the middle frame as the keyframe, which is repeated to form a static “video” containing no actual motion (Siam, 2 Sep 2025). If a model still predicts the referred object on this repeated-frame video, the paper interprets that as evidence of reliance on static cues.

The Reverse video probe tests sensitivity to motion order or the arrow of time. The original video is temporally reversed, and the referring expression is automatically rewritten to describe the reversed video. GPT-4o is prompted with: “Taken an input motion referring expression as <EXP> corresponding to an original video, can you convert it to the motion expression describing what will occur in the reverse version of the same video? Output the new expression only.” The paper gives examples such as “turn and walk away from us” becoming “turn and walk towards us,” and “pulling” becoming “pushing” (Siam, 2 Sep 2025). The resulting reverse expressions are manually inspected, and ambiguous examples are discarded.

The Multi-video layout with single frame probe is described as a stronger version of the motion-existence test. It creates a split-screen layout where one side shows the original video and the other side shows the repeated-keyframe video. The appendix describes three layouts, including the two split-screen variants used to avoid spatial bias: original-left/modified-right and modified-left/original-right (Siam, 2 Sep 2025). This setting is the primary test of whether a model can identify the true-motion stream when static appearance is nearly matched.

The Multi-video layout with reverse video probe is the strongest test for motion-order understanding. A split-screen layout contains the original video on one side and the reversed video on the other, with the expression matched to one of them. Because the static content may be nearly identical, the distinguishing factor is the direction or order of motion (Siam, 2 Sep 2025).

4. Evaluation protocol and metrics

The benchmark uses standard video object segmentation metrics: region similarity $\mathcal{J}$, contour accuracy $\mathcal{F}$, and their average (Siam, 2 Sep 2025). The paper states: “We use the standard evaluation metrics for the region similarity, $\mathcal{J}$, the contour accuracy, $\mathcal{F}$, and their average, $\mathcal{J{paper\_content}F}$.” No additional thresholding protocol is specified in the main text.

For the repeated-static-frame-only setting, the expected correct output is all background. To avoid trivial domination by background pixels, the appendix defines a modified background IoU metric, $\mathcal{F}$0, which relabels the original foreground pixels as “modified background” and computes IoU only over those pixels, ignoring the original background (Siam, 2 Sep 2025). The paper states that this metric “provides a better measure to evaluate the false positives appearing in that static keyframe.”

The reported MoCentric-Bench evaluation variants derived from MeVIS val_u are:

Variant	Description
Single frame	Repeated keyframe-only modified video
val_u {paper_content} Single frame	Split-screen original + repeated keyframe
Reverse	Reversed video + reversed expression
val_u {paper_content} Reverse	Split-screen original + reversed video

The extracted notation is garbled in the source text, but these interpretations are explicitly given there (Siam, 2 Sep 2025).

5. Empirical findings

The paper evaluates prior referring video segmentation methods, current video MLLMs, strong single-image baselines, and an adapted Sa2VA model (Siam, 2 Sep 2025). The compared methods include ReferFormer, LMPM, LISA, TrackGPT, VISA, VidGLAMM / VideoGLAMM, and Sa2VA. The single-image baselines introduced by the paper are MLLM+S2 and MLLM+S2$\mathcal{F}$1, where the base MLLM is Qwen2.5-VL-7B and segmentation is produced with SAM 2.0. The adapted model is Sa2VA$\mathcal{F}$2, a motion-centric LoRA adaptation of Sa2VA (Siam, 2 Sep 2025).

A central empirical argument is that strong single-image baselines rival or exceed prior video methods on standard benchmarks. On RefDAVIS-17, the paper reports $\mathcal{F}$3 for VideoGLAMM, $\mathcal{F}$4 for MLLM+S2, $\mathcal{F}$5 for MLLM+S2$\mathcal{F}$6, $\mathcal{F}$7 for Sa2VA, and $\mathcal{F}$8 for Sa2VA$\mathcal{F}$9 (Siam, 2 Sep 2025). On MeVIS val, it reports $\mathcal{J{paper\_content}F}$0 for VideoGLAMM, $\mathcal{J{paper\_content}F}$1 for MLLM+S2, $\mathcal{J{paper\_content}F}$2 for MLLM+S2$\mathcal{J{paper\_content}F}$3, $\mathcal{J{paper\_content}F}$4 for Sa2VA, and $\mathcal{J{paper\_content}F}$5 for Sa2VA$\mathcal{J{paper\_content}F}$6 (Siam, 2 Sep 2025). The significance attached to these numbers is that a grounding pipeline using only one selected frame can be competitive with full video models, which the paper takes as evidence that standard benchmarks do not strongly require temporal understanding.

On MoCentric-Bench itself, the paper reports the following results:

Method	val_u	split-screen single-frame	Reverse	split-screen reverse
LMPM	37.2	22.6	35.0	21.7
VidGLAMM	48.2	21.6	53.9	34.0
Sa2VA	58.9	28.5	61.1	37.0
MLLM+S2	52.0	26.5	48.5	25.8
MLLM+S2$\mathcal{J{paper\_content}F}$7	57.4	28.1	53.4	31.1
Sa2VA$\mathcal{J{paper\_content}F}$8	60.5	31.1	65.1	42.0

Across methods, performance drops dramatically—described in the source as roughly by half—when moving from the standard or simply reversed settings to the motion-centric split-screen layouts (Siam, 2 Sep 2025). This is the benchmark’s strongest quantitative finding: current visual grounding systems remain strongly biased toward static appearance information.

For the pure single-frame-only probe, the appendix reports $\mathcal{J{paper\_content}F}$9 values of 5.5 for LMPM, 8.8 for VideoGLAMM, 13.2 for MLLM + SAM 2.0, and 13.1 for MLLM + SAM 2.0$\mathcal{J}$0 (Siam, 2 Sep 2025). These low values are interpreted as showing that models still hallucinate foreground segmentation on a static video with no real motion target.

6. Baselines, adaptation, and analysis

The main baseline pipeline uses Qwen2.5-VL-7B-Instruct with the prompt: “Locate the <EXP>, output its bbox coordinates using JSON format.” For the basic baseline, the first frame is used; if that frame does not yield a parseable box, the procedure advances to the first frame that does. The MLLM + SAM 2.0$\mathcal{J}$1 variant uses the automatically selected keyframe instead (Siam, 2 Sep 2025). An appendix ablation on val_u reports 52.0 for First Frame, 53.3 for Last Frame, and 57.4 for KeyFrame, supporting the choice of selected keyframe as the strongest static baseline (Siam, 2 Sep 2025).

To avoid spatial bias in multi-video layouts, the authors evaluate both original-left/modified-right and modified-left/original-right configurations and combine predictions for SAM 2.0 initialization (Siam, 2 Sep 2025). All experiments are reported as having used an A6000 GPU.

The paper also proposes motion-centric low-rank adaptation (LoRA) on the vision encoder of Sa2VA using supervised synthetic motion-centric training data derived from the MeVIS training subset. The synthetic training data uses the same multi-video layouts, specifically original + static keyframe and original + reverse (Siam, 2 Sep 2025). Reported hyperparameters are rank $\mathcal{J}$2, $\mathcal{J}$3, initialization from “ByteDance/Sa2VA-8B”, learning rate $\mathcal{J}$4, and dataset sampling ratio 1:1 between original MeVIS train and synthetic motion-centric train (Siam, 2 Sep 2025).

This adaptation improves MoCentric-Bench performance from 28.5 to 31.1 on split-screen single-frame and from 37.0 to 42.0 on split-screen reverse, compared with original Sa2VA (Siam, 2 Sep 2025). At the same time, the paper notes that the improvement in the gap between standard and motion-centric settings is only around 1%, indicating that simple adaptation is helpful but insufficient.

The authors further analyze the 793 segmentation-expression pairs in the single-frame-based evaluation by separating them into a Motion Group with less than 2% false positives and a Static Group with more than 2% false positives (Siam, 2 Sep 2025). They report that 380 out of 793 pairs fall into the second group. Their GPT-4o-based analysis characterizes the Motion Group as having richer dynamic verb phrases, more multi-step actions, transitions, and directional movement, whereas the Static Group more often contains abstract or static expressions, static poses, and simpler states or summaries (Siam, 2 Sep 2025). Fine-grained property analysis associates the Motion Group more with multi-step action: 56.1 vs 46.2 and dynamic verb phrase: 26.5 vs 18.2, while the Static Group is more associated with multi-object interaction: 44.9 vs 41.3, category name: 58.3 vs 52.0, and color: 10.3 vs 6.6 (Siam, 2 Sep 2025).

7. Position in the benchmarking landscape, limitations, and implications

MoCentric-Bench is situated as a corrective to video grounding benchmarks that can be solved by single-frame shortcuts. In this respect it differs from MC-Bench, which targets multi-context visual grounding across image pairs and studies cross-image reasoning and localization rather than motion dependence within videos (Xu et al., 2024). MC-Bench focuses on localizing instances across multiple images from open-ended prompts, whereas MoCentric-Bench focuses on enforcing the use of motion-language interaction in referring video segmentation (Xu et al., 2024, Siam, 2 Sep 2025). This suggests that the two benchmarks probe different failure modes of multimodal systems: cross-image context integration in one case, and genuine temporal grounding in the other.

The paper explicitly describes its contribution as initial groundwork and states that future work should build motion-centric datasets under a more principled mathematical framework (Siam, 2 Sep 2025). Additional limitations visible in the source include dependence on GPT-4o plus manual filtering for reverse-expression generation, the fact that only a subset of MeVIS examples can be reversed unambiguously, the inheritance of constraints from MeVIS because the benchmark is synthesized from existing data, and the restriction of adaptation to LoRA tuning of the vision encoder (Siam, 2 Sep 2025).

The broader implication drawn in the paper is that high scores on existing video grounding benchmarks do not reliably indicate motion understanding. Strong single-image baselines can be competitive with state-of-the-art video methods on standard benchmarks, yet all tested systems degrade sharply under the controlled motion-centric probes of MoCentric-Bench (Siam, 2 Sep 2025). The benchmark therefore functions less as a general-purpose segmentation leaderboard than as a diagnostic instrument for exposing reliance on static cues in pixel-level video grounding.