Dyn-Bench: 4D Dynamic Scene Benchmark

• Dyn-Bench is a benchmark that evaluates multimodal models on dynamic spatio-temporal reasoning and object grounding in 4D scenes.
• It integrates static image understanding with dynamic tracking by testing models on 1,000 videos annotated with visual question answering and grounding pairs.
• Empirical results reveal a performance gap between spatio-temporal reasoning and grounding, emphasizing the need for structured interventions like ST-TCM and Mask-Guided Fusion.

to=arxiv_search.search 神彩争霸.json {"query":"(Huang et al., 13 Mar 2026) Dyn-Bench Thinking in Dynamics benchmark multimodal LLMs 4D world", "max_results": 5, "sort_by":"submittedDate", "sort_order":"descending"} to=arxiv_search.search 天天中彩票网络ള്ളി to=arxiv_search.search ฝ่ายขายออนไลน์ Dyn-Bench is a benchmark for evaluating whether multimodal LLMs (MLLMs) can perceive, track, localize, and reason about dynamic scenes in a physical $4\text{D}$ world rather than merely recognize static visual content. Introduced in “Thinking in Dynamics: How Multimodal LLMs Perceive, Track, and Reason Dynamics in Physical 4D World,” it targets the joint problem of spatio-temporal reasoning and dynamic object grounding through a benchmark built from real-world and synthetic video sources, comprising $1{,}000$ videos, $7{,}000$ visual question answering (VQA) pairs, and $3{,}000$ dynamic object grounding pairs (Huang et al., 13 Mar 2026).

1. Conceptual scope

Dyn-Bench is organized around the claim that strong static-image understanding does not imply strong dynamic understanding. The benchmark therefore operationalizes “thinking in dynamics” as the ability to maintain coherent object identities over time, understand motion and interaction, reason about scene evolution and camera movement, and align language with localized dynamic evidence (Huang et al., 13 Mar 2026).

The benchmark is structured around three complementary levels of dynamic scene understanding. These levels couple scene-level reasoning with object-level grounding so that performance cannot be reduced to fluent verbal description alone. A central design choice is that a model should not only answer dynamic questions correctly, but should also localize the relevant dynamic entities across time.

Level	Core focus	Subtask categories
Dynamic Inter-Object Perception	interactions and relations among moving objects	Act. Obj. Desc.; Move. Obj. Temp. Dyn.; Spatial Rel. Change
Dynamic Object–Scene Tracking	following an object through an evolving environment	Mov. Patterns Traj.; Spatial Rel. Comp.; Scene Focus Dyn.
Dynamic Camera–Object Reasoning	disentangling object motion from camera motion	Cam. Motion Orient.; Cam-Obj. Interaction; Temp. Obj. Visual Change

The table labels are abbreviated in the benchmark results, but the accompanying prose clarifies that they cover action and interaction description, movement and temporal dynamics, changes in spatial relations, motion patterns and trajectories, object-scene evolution, scene-focused dynamics, camera motion orientation, camera-object interaction, and temporal visual change. This organization makes Dyn-Bench object-centric rather than purely scene-centric, and it is intended to expose failures in temporal continuity, interaction reasoning, and grounding consistency that static visual benchmarks do not measure (Huang et al., 13 Mar 2026).

2. Construction and curation pipeline

Dyn-Bench is built from eight source datasets: the $2\text{D}$ video segmentation datasets DAVIS, SA-V, DynPose-100K, and YouTube-VIS, together with the $4\text{D}$ dynamic-scene datasets DynamicReplica, PointOdyssey, Spring, and Total-Recon. The benchmark construction process starts from $38{,}507$ source videos, narrows them to $2{,}026$ filtered candidates, and then selects $1{,}000$ benchmark videos through human review (Huang et al., 13 Mar 2026).

The curation pipeline is multi-stage. It first computes motion and quality statistics such as blur, frame rate, I-frame count, motion vector magnitude, and motion vector variance. It then derives geometric features using VGGT and UniDepth-V2, which estimate per-frame camera intrinsics, extrinsics, and depth maps. Dynamic object coverage is estimated next; for $4\text{D}$ datasets that lack masks, masks are inferred or refined with Qwen2.5-VL and Sa2VA. An MLLM then answers $1{,}000$0 structured diagnostic questions, producing semantic and motion-aware descriptors. These are aggregated into $1{,}000$1-dimensional features and scored by a random forest regressor trained on $1{,}000$2 manually annotated videos. The final stage applies VLM-assisted refinement for semantic coherence, realism, and motion validity before human selection (Huang et al., 13 Mar 2026).

After filtering, missing annotations are completed using Sa2VA, ViPE, and Qwen3 technical report pipelines, followed by human quality control. The review procedure checks video quality, mask consistency, VQA correctness, dynamic object categories, temporal mask coherence, object identity consistency, and alignment between generated reasoning tasks and the underlying visual evidence. This curation strategy is designed to ensure that Dyn-Bench evaluates dynamic understanding rather than artifacts of poor geometry, unstable masks, or inconsistent question generation (Huang et al., 13 Mar 2026).

3. Task formulation and evaluation

Dyn-Bench uses a dual evaluation protocol. For reasoning, it adopts multiple-choice VQA, with Accuracy (ACC) computed by exact match of the selected answer. For grounding, it uses temporally coherent instance segmentation masks rather than bounding boxes, and evaluates them with the video object segmentation metric that averages region similarity $1{,}000$3 and contour or boundary accuracy $1{,}000$4 (Huang et al., 13 Mar 2026).

The benchmark construction and later improvement methodology are both tied to a formal intermediate representation called the Spatio-Temporal Textual Cognitive Map (ST-TCM). In the supplementary description, each video is sampled at $1{,}000$5 FPS with RGB-D frames and instance masks, geometry is reconstructed using VIPE, and per-object motion attributes are derived from reconstructed $1{,}000$6 trajectories. Temporal differencing is used to define velocity and acceleration, for example

$1{,}000$7

and

$1{,}000$8

The resulting representation includes object size and position, motion direction, velocity, acceleration, relative distance and orientation, and interaction type, which are then textualized into a coherent spatio-temporal narrative (Huang et al., 13 Mar 2026).

An important feature of the evaluation setup is what it does not do. Dyn-Bench does not define a new universal composite consistency metric that merges reasoning and grounding into a single scalar. Instead, it evaluates VQA accuracy and dynamic grounding quality separately. This separation is central to the benchmark’s diagnostic value, because it reveals when a model is verbally plausible but poorly grounded, or conversely when it can localize objects without achieving strong spatio-temporal reasoning (Huang et al., 13 Mar 2026).

4. Empirical profile and observed failure modes

Dyn-Bench evaluates three families of models: general MLLMs, spatial MLLMs, and region-level MLLMs. All models are tested in zero-shot mode with default instruction templates. General and spatial MLLMs are evaluated only on reasoning, whereas region-level models are evaluated on both reasoning and grounding (Huang et al., 13 Mar 2026).

The benchmark’s central empirical finding is that current MLLMs do not simultaneously maintain strong performance in both spatio-temporal reasoning and dynamic object grounding. On VQA, Qwen3-VL-235B reaches the highest reported average accuracy at $1{,}000$9, while Gemini-2.5 Pro reaches $7{,}000$0 and GPT-5 reaches $7{,}000$1. Among spatial MLLMs, SpaceR-7B reaches $7{,}000$2. Among region-level models on reasoning, UniPixel-7B leads with $7{,}000$3. On grounding, however, the best model is Sa2VA-InternVL2.5-8B, with $7{,}000$4 $7{,}000$5, $7{,}000$6 $7{,}000$7, and $7{,}000$8 averaged $7{,}000$9 (Huang et al., 13 Mar 2026).

This ranking divergence is one of Dyn-Bench’s most important results. The strongest reasoning models are not the strongest grounding models, and strong grounding does not guarantee top spatio-temporal reasoning. The benchmark therefore exposes a mismatch that would be obscured by a purely textual evaluation or a purely localization-based benchmark (Huang et al., 13 Mar 2026).

The error analysis further shows that fluent reasoning traces can be physically inconsistent. One cited failure case involves GPT-4o producing a linguistically plausible explanation of when a white car catches up to a pedestrian, but basing the answer on apparent size changes rather than metric motion reasoning. The paper groups such errors into temporal reasoning errors, spatial grounding errors, and relational reasoning errors. It also reports broader failure patterns involving inconsistent motion interpretation, weak tracking, poor interaction reasoning, mismatch between textual answers and localized evidence, and confusion between object motion and camera motion. Camera-object categories are generally harder than inter-object or object-scene reasoning, especially for models without strong geometric or region-level priors (Huang et al., 13 Mar 2026).

5. Structured enhancement methods

Dyn-Bench is not only an evaluation benchmark; it is also used to study structured interventions that improve dynamic understanding. The paper contrasts conventional prompting methods such as chain-of-thought and caption-based hints with two stronger approaches: Mask-Guided Fusion and ST-TCM (Huang et al., 13 Mar 2026).

Mask-Guided Fusion is a visual-side intervention in which the model receives both raw frames and segmentation-mask overlays. The benchmark reports that for Qwen3-VL-8B, raw video yields Inter-Object $3{,}000$0, Object-Scene $3{,}000$1, Camera-Object $3{,}000$2, and average $3{,}000$3. Masked Frames Only remains at $3{,}000$4 average, indicating negligible change. Mask-Guided Fusion raises the scores to $3{,}000$5, $3{,}000$6, $3{,}000$7, and average $3{,}000$8, a gain of $3{,}000$9 over raw video. The largest gains appear in Inter-Object and Camera-Object reasoning, which suggests that localization cues help most when dynamic entity tracking and camera-motion disentanglement are central (Huang et al., 13 Mar 2026).

ST-TCM is the stronger textual-side intervention. For Qwen3-VL-32B on VQA, the benchmark reports average accuracy of $2\text{D}$0 without TCM, $2\text{D}$1 with temporal semantics alone, $2\text{D}$2 with motion alone, $2\text{D}$3 with spatial geometry alone, and $2\text{D}$4 with the full temporal-motion-spatial configuration, a gain of $2\text{D}$5. For Sa2VA-InternVL2.5-8B on grounding, the no-TCM baseline is $2\text{D}$6 across Inter-Object, Object-Scene, and Camera-Object, while the full temporal-motion-spatial configuration improves these to $2\text{D}$7, with the largest improvement in Camera-Object grounding (Huang et al., 13 Mar 2026).

A consistent conclusion follows from these ablations: temporal cues alone provide limited benefit, whereas motion and geometry are the key ingredients, and their coupling yields the strongest gains. This suggests that the bottleneck in dynamic multimodal intelligence is not merely longer temporal context, but the absence of structured representations of object motion, geometry, and relational change (Huang et al., 13 Mar 2026).

6. Benchmark position, naming, and limitations

Dyn-Bench is positioned as a benchmark that jointly assesses inter-object, object-scene, and camera-object dynamics together with dynamic object grounding across mixed real and synthetic data at metric scale. Its distinctive feature is the coupling of language-side spatio-temporal reasoning with mask-based object grounding, which makes it a benchmark for coherent dynamic interpretation rather than generic video question answering alone (Huang et al., 13 Mar 2026).

The name can be confused with other benchmark efforts. Dyn-Bench is distinct from Dynabench, which is an open-source platform for dynamic dataset creation and model benchmarking in NLP (Kiela et al., 2021), and from Dynaboard, the evaluation-as-a-service layer integrated with that ecosystem (Ma et al., 2021). The overlap is nominal rather than conceptual: Dynabench and Dynaboard address dynamic benchmarking workflows in NLP, whereas Dyn-Bench targets multimodal spatio-temporal understanding in video and $2\text{D}$8 scene data.

The benchmark also has explicit limitations. Its annotations depend partly on reconstructed geometry, inferred masks, and rule-based textualization through ST-TCM. Its VQA task uses multiple-choice format rather than open-ended generation. The provided description does not specify a train/validation/test split. More broadly, the reported results show that rapid motion, occlusion, viewpoint change, and complex interactions remain difficult even for the strongest models. A plausible implication is that future progress will require tighter integration of temporal semantics, geometry, motion, and object-level grounding rather than incremental improvements to static visual-language modeling alone (Huang et al., 13 Mar 2026).