- The paper introduces the Causal-Generative Dual-Judge benchmark, combining flattened-video causal questioning with future-event generation to evaluate explicit understanding and implicit prediction.
- The paper finds a major Perception-Prediction Gap: Wan-2.2 and HunyuanVideo score only 0.7% and 1.5% on explicit causal understanding despite achieving 41.5% and 42.5% on generative action.
- The paper shows that Sora-2 and Veo-3.1 perform more consistently, but all models struggle more with physical science than social dynamics and often verbalize correct reasoning without rendering it accurately.
Motivation and problem statement
The paper examines whether video generative models can serve as reasoning engines rather than merely as high-fidelity synthesizers. The authors formalize this question as the "Thinking in Video" paradigm, in which video functions as a medium for constructing, extending, and verifying causal thought rather than as a final output artifact. The central concern is that a visually convincing rollout does not imply causal understanding: a model may render shattering glass or a falling object because it has internalized physical law, or because it reproduces memorized high-frequency visual associations. Existing evaluation practices cannot distinguish these cases, since distributional metrics such as FVD and Inception Score reward perceptual plausibility while ignoring whether generated futures obey scene mechanics, and static multimodal benchmarks probe symbolic QA without requiring evolving video answers. This structural bifurcation of visual fidelity from semantic logic conceals what the paper terms Perception-Prediction Dissonance: a model may know an outcome yet fail to render it consistently.
The Causal-Generative Dual-Judge framework
To audit world-model consistency from both sides, the paper introduces the Causal-Generative Dual-Judge (CGDJ), comprising a benchmark and an automated evaluation pipeline with two complementary modules.
Explicit Causal Perception tests whether a generator can read a video scenario as a reasoning problem. Because current video generators accept only static image inputs, the authors propose the Flatten Temporal Video strategy: 70 uniformly sampled frames are concatenated into a 7×10 spatial grid, the textual query is rasterized into pixel space above the grid, and the composite is resized to 1280×720. This converts a video-question pair into a single image on which the generator must produce a directly judgeable answer, forcing it to reveal recognized causal progression rather than statistical continuation. The data derive from all 900 Video-MME videos spanning short, medium, and long durations across six domains (Knowledge, Life Record, Sports Competition, Artistic Performance, Film & Television, Multilingual).
Implicit Generative Prediction tests whether a model can enact its reasoning by rendering the causal consequence. The benchmark curates 600 videos balanced between Natural Sciences (drawn from The Sound of Water, Physion, and Physics-IQ; covering mechanics, life sciences, materials/structures, fields/energy) and Sociology & Humanities (from TLD, Kinetics, Ego4D, and web videos). Expert annotators identify a Causal Inflection Point in each video; pre-event frames (7 keyframes flattened horizontally into a "visual arrow of time") plus a Gemini-3-Pro semantic description condition generation of the post-event segment, which is scored against the ground-truth continuation.
Evaluation is fully automated. For explicit tasks, audio is transcribed via whisper-large-v2 and Gemini-3-Pro adjudicates causal correctness against ground truth. For implicit tasks, Gemini-3-Pro scores generations in [0,1] along Semantic Alignment, Reference Consistency, and Physical Validity. Human alignment is strong: Pearson correlation of 0.8205 and mean absolute difference of 0.1468 between the judge and three expert raters, supporting the reliability of the automated protocol.
Main results
The experiments cover four generators — open-source Wan-2.2-14B and HunyuanVideo-1.5, closed-source Veo-3.1 and Sora-2 — benchmarked against MLLMs under both native video input and the flattened format.
| Model |
Explicit Causal Understanding (%) |
Implicit Generative Action (%) |
| Wan-2.2-14B |
0.7 |
41.5 |
| HunyuanVideo-1.5 |
1.5 |
42.5 |
| Veo-3.1 |
49.9 |
58.9 |
| Sora-2 |
51.2 |
61.6 |
Three findings stand out. First, flattening preserves sufficient information for reasoning: Gemini-3-Flash retains roughly 89% of its native-video accuracy (74.1% vs. 83.0%) under the flattened protocol, validating the medium itself. Second, a sharp divide separates closed- and open-source generators on explicit perception. Sora-2 (51.2%) and Veo-3.1 (49.9%) perform on par with Qwen3-VL-8B, whereas Wan-2.2 (0.7%) and HunyuanVideo (1.5%) collapse entirely, which the authors attribute to absent instruction-following alignment. Third, and most consequential, juxtaposing the two evaluations exposes the Perception-Prediction Gap: Wan-2.2 achieves 41.5% on generative action despite near-zero explicit understanding, indicating statistical pixel mimicry that renders the texture of causal events without comprehending their logic. In contrast, Sora-2's synchronized profile (51.2% explicit, 61.6% implicit) suggests that scaling toward genuine world simulators increasingly grounds generative action in explicit causal grasp. Notably, all models score higher on Sociology & Humanities than Natural Sciences (e.g., Sora-2 at 65.6% vs. 57.7%), implying that rigid physical laws remain harder to internalize than social dynamics patterns.
Audio-visual misalignment
A diagnostic analysis decomposes generation accuracy into audio-only versus video-only channels under identical inputs. Verbalized answers are consistently more accurate than rendered ones: models reliably articulate correct causal logic while failing to physically simulate it. The authors characterize this as a "theoretical expert but practical novice" profile, concluding that current systems lean on linguistic priors rather than operating through a robust internal world model. This result directly challenges the "world simulator" narrative attached to modern video generators.
Ablations over grid density reveal an inverted-U curve peaking at the default configurations. Increasing the explicit grid to 8×12 degrades performance (Veo-3.1 falls from 49.9% to 43.4%; Sora-2 to 47.4%) due to visual compression within cells, while sparser layouts lack temporal resolution for longer sequences. Similarly, 7 conditioning keyframes outperform both sparse (1, 4) and dense (10) settings for implicit prediction, reflecting a trade-off between motion ambiguity at low temporal density and contextual redundancy at high density. These results indicate that the reported numbers depend on a specific, empirically optimized input encoding, and alternative encodings could shift absolute scores.
Limitations and open questions
Several constraints qualify the conclusions. The Flatten Temporal Video strategy is itself a workaround for architectures that cannot natively consume interleaved video-text prompts; the near-total failure of open-source models on explicit tasks may partly reflect this mismatched interface rather than absent causal knowledge alone, although the MLLM controls mitigate this concern. Evaluation relies on Gemini-3-Pro as both data annotator and judge, introducing potential self-evaluation bias despite the reported human correlation of 0.82. The benchmark's inflection points are manually annotated, and the implicit task conditions generation on only 7 keyframes plus a text description, so models may be handicapped relative to native video conditioning. Open questions include whether instruction-tuning or interleaved-input architectures can close the gap for open-weight generators, whether the audio-visual misalignment persists when models are trained to bind verbalized physics to rendering, and whether the observed superiority on social dynamics over physical mechanics reflects data distribution or a deeper asymmetry in learnability.
Conclusion
This paper reframes video-based reasoning as Thinking in Video and supplies CGDJ, a dual-judge benchmark that jointly audits explicit causal perception and implicit generative prediction. Its principal empirical contribution is quantifying the Perception-Prediction Gap: plausible dynamics can coexist with near-zero causal comprehension in open-source models, while frontier closed-source systems exhibit early but limited alignment between reasoning and generation, compounded by systematic audio-visual misalignment. The work argues for shifting video-generation evaluation from perceptual fidelity toward causal consistency and provides code and data to support that shift.
