IntPhys 2: Realistic Test of Intuitive Physics

• IntPhys 2 is a synthetic video benchmark designed to test AI’s intuitive physics by distinguishing physically possible from impossible events.
• It employs photorealistic Unreal Engine scenes with dynamic lighting, realistic occlusion, and moving camera viewpoints to enhance the evaluation’s realism.
• The benchmark assesses core physical principles—Permanence, Immutability, Spatio-Temporal Continuity, and Solidity—revealing a significant performance gap between humans and current AI models.

IntPhys 2 is a synthetic video benchmark for evaluating intuitive physics understanding in AI systems under controlled but substantially more complex conditions than the original IntPhys benchmark. It is framed as a violation-of-expectation test in which models must differentiate physically possible from physically impossible events across four core principles related to macroscopic objects: Permanence, Immutability, Spatio-Temporal Continuity, and Solidity. The benchmark was introduced as a more challenging and more realistic successor to IntPhys, with photorealistic Unreal Engine scenes, richer backgrounds, realistic lighting and shadows, more natural occlusion events, and moving-camera viewpoints. Reported results show a large gap between human and machine performance: humans achieve near-perfect accuracy, whereas the tested multimodal LLMs and predictive video models remain close to chance on the main and held-out sets (Bordes et al., 11 Jun 2025).

1. Origins and rationale

IntPhys 2 is best understood in relation to the original "IntPhys" benchmark, which tested whether a system could assign higher plausibility scores to possible videos than to carefully matched impossible ones. The original benchmark focused on three core concepts for macroscopic solid objects—object permanence, shape constancy, and spatio-temporal continuity—and emphasized matched possible/impossible constructions intended to minimize low-level statistical bias (Riochet et al., 2018). A parallel line of work argued that intuitive-physics evaluation should be grounded in the developmental-psychology notion of violation of expectation, using controlled probe videos and model surprise rather than only generic future-prediction error (Piloto et al., 2018).

The motivation for IntPhys 2 is that the original IntPhys setup had become too easy for newer predictive models and was limited by simplistic scenes and weak occlusion structure. IntPhys 2 therefore expands the earlier design into photorealistic Unreal Engine environments with richer backgrounds, realistic lighting and shadows, more natural occlusion events, and moving-camera viewpoints that mimic how humans actually encounter occlusion in the world. The benchmark is explicitly intended to test short-term memory, occlusion tracking, robustness to viewpoint changes, and the ability to reason about hidden objects and physical interactions over longer time spans rather than merely detect superficial visual cues (Bordes et al., 11 Jun 2025).

A central design decision is the use of the violation-of-expectation paradigm rather than ordinary action recognition or captioning. The benchmark asks whether a model treats impossible events as more surprising or less plausible than matched possible events. This places IntPhys 2 in the same family as IntPhys-style physical-plausibility datasets, while increasing realism and temporal difficulty (Bordes et al., 11 Jun 2025).

2. Dataset organization and scene construction

IntPhys 2 organizes each scene as a quadruplet of videos: two possible and two impossible. The quadruplet structure is designed so that a slight change can flip a possible version into the impossible version of another, which suppresses shortcuts based on local appearance and makes the task more diagnostic of physical reasoning. The benchmark is generated in Unreal Engine, using assets from the Unreal Marketplace and Fab, and includes textured, object-rich environments, dynamic shadows, realistic lighting, and varying camera motion (Bordes et al., 11 Jun 2025).

Two camera regimes are emphasized. In fixed camera scenes, occlusion is created by an object moving through the scene. In moving camera scenes, occlusion occurs as a consequence of observer motion. The latter is presented as closer to human perception because it mimics looking away and then back, or moving around an obstacle (Bordes et al., 11 Jun 2025).

The dataset is split as follows:

Split	Scenes	Videos
Debug set	5	60
Main set	253	1,012
Held-out set	86	344

The main set is further divided into easy, medium, and hard difficulty levels. Easy scenes use simple backgrounds and colorful shapes; medium scenes use more diverse backgrounds and textured objects; hard scenes use realistic objects and complex environments. The benchmark is deliberately not massive, because the authors note that generating many nearly repeated scenes would not actually increase diversity and would raise overfitting risk (Bordes et al., 11 Jun 2025).

A later multimodal-language-model evaluation describes IntPhys 2 as containing 1,416 photorealistic videos, and separately refers to 1,012 videos in the probing section. These counts are consistent with the full benchmark total and the main evaluation split, respectively (Ballout et al., 22 Jul 2025).

3. Physical principles and operationalization

IntPhys 2 evaluates four physical principles grounded in developmental psychology and core-knowledge theory: Permanence, Immutability, Spatio-Temporal Continuity, and Solidity (Bordes et al., 11 Jun 2025).

Permanence tests whether a model expects objects to continue to exist over time, even when hidden from view. Typical scenarios involve an object hidden behind an occluder or container and later revealed; impossible versions violate persistence by making the object vanish, fail to reappear, or otherwise break expected continuity.

Immutability tests whether a model expects an object to maintain its intrinsic properties—shape, texture, or structure—when occluded or when the viewpoint changes. The violation is not disappearance but impossible change in object identity or material properties. In this sense, IntPhys 2 extends the original IntPhys concern with shape constancy into a broader notion of property stability.

Spatio-Temporal Continuity tests whether objects move smoothly and continuously through space and time. Impossible videos violate continuity by forcing reappearance at an inconsistent location or by introducing discontinuous motion incompatible with the visible trajectory.

Solidity tests whether solid objects block one another and whether collisions alter motion. Appendix examples include FixedJumpSolidity, SolidityFallingFlat, SphereFallingDownSolidity, FixedMarryPoppins, BoxSolidity, Scaffolding, CameraSolidity, and JumpSolidity. These scenes include events in which rolling balls or other moving objects should be blocked by boxes, scaffolds, or other obstacles; impossible videos allow objects to pass through barriers, ignore collisions, or reveal occluders in physically implausible ways (Bordes et al., 11 Jun 2025).

A later study of multimodal LLMs summarizes the same four principles as Object permanence, Object immutability, Spatio-temporal continuity, and Solidity, and uses them to characterize IntPhys 2 as a photorealistic violation-of-expectation benchmark for binary plausibility judgment (Ballout et al., 22 Jul 2025).

4. Evaluation protocol and scoring

IntPhys 2 supports three evaluation styles: human evaluation, MLLM evaluation, and prediction-based evaluation (Bordes et al., 11 Jun 2025).

For human evaluation, annotators judge plausibility on a 1–4 Likert scale after watching the videos. Each of the 1,416 videos is rated by 3 annotators, with randomized order and a maximum of 96 videos per annotator. Annotators are shown 10 plausible videos first to calibrate them to the simulator style, and final human performance is computed by majority vote.

For MLLM evaluation, the task is simplified to binary plausibility judgment because current models cannot easily process long video contexts or retain long-term memory in the same way as humans. The benchmark paper reports sensitivity to prompt format and frame count, using prompts that request open-ended yes/no answers, forced binary digit output, or yes/no with stronger instruction wording; it also reports the best accuracy across runs because of prompt sensitivity and output-format variance (Bordes et al., 11 Jun 2025).

For predictive video models, IntPhys 2 uses a violation-of-expectation-style surprise score. Given a video $V$ with $N$ frames and a model that processes windows of length $M$, each window $w$ is partitioned into context length $C$ and prediction target $T=M-C$. The benchmark defines

$$Surprise_w = d\left(p\left(f\left(V_{w:w+C}\right)\right), f'\left(V_{w+C:w+M}\right) \right),$$

with aggregated scores

$$AvgSurprise =\frac{1}{\#W} \sum_{w \in W} Surprise_w$$

and

$MaxSurprise =\max_{w\in W}\left(Surprise_w\right).$

Average surprise is used mainly for pairwise comparisons, whereas max surprise is more appropriate for single-video classification. The protocol is adapted so that, as the sliding window advances, every frame can be predicted regardless of window size (Bordes et al., 11 Jun 2025).

A subsequent MLLM study uses IntPhys 2 as a direct plausibility judgment task with videos sampled at 1 FPS, 10 frames per video for a 10-second clip, and evaluates three prompting protocols: free generation, greedy decoding, and Chain-of-Thought prompting. For free generation and CoT, results are averaged over three random seeds, using temperature = 0.6, top-p = 0.9, and top-k = 40 (Ballout et al., 22 Jul 2025).

5. Reported results and observed failure modes

The central empirical result is that current deep models remain far below human performance on IntPhys 2. On the main set, the benchmark paper reports the following best accuracies for the tested systems; the same study also reports held-out performance (Bordes et al., 11 Jun 2025).

System	Main set	Held-out
GPT-4o	53.75%	53.19%
Qwen-VL 2.5	52.27%	49.12%
Gemini 1.5 Pro	52.27%	52.10%
Gemini 2.5 Flash	55.63%	56.10%
VideoMAEv2	53.75%	53.49%
Cosmos-Predict-4B	49.41%	48.84%
V-JEPA + RoPE	53.75%	54.65%
V-JEPA 2	57.51%	56.40%
Human	96.44%	92.44%

By condition, Permanence is the easiest principle for both humans and models, whereas Solidity tends to be hard for models. Humans remain consistently high across both fixed and moving cameras. Some subset scores are higher—for example, V-JEPA 2 reaches 75.00% on continuity in moving-camera scenes, and Gemini 2.5 Flash performs best on permanence and some immutability subsets—but the benchmark authors caution that such results come from small subsets and should not be overinterpreted (Bordes et al., 11 Jun 2025).

The paper also reports several ablations. Prompt sensitivity is substantial for MLLMs, accuracy varies with output format and prompt wording, and some models fail to follow the requested format. Frame count sensitivity is also notable: adding more frames often hurts rather than helps, which suggests poor long-context use. The debug set shows that even nearly identical videos can trigger different predictions because of small rendering or compression artifacts. A comparison with an earlier prediction protocol indicates that the low results are not an artifact of the revised evaluation setup (Bordes et al., 11 Jun 2025).

Qualitative analysis emphasizes memory and occlusion failures. V-JEPA surprise curves often respond to motion but not reliably to the physically critical event of reappearance after occlusion. Cosmos struggles to retain small objects in latent reconstructions; the ball can fade from context prediction and fail to reappear properly. The benchmark authors stress that many samples require remembering objects over nearly 200 frames, which exceeds the effective temporal handling of many current video models (Bordes et al., 11 Jun 2025).

A later MLLM-specific study reaches a similar conclusion. It reports that all evaluated models remain at chance or only slightly above chance on IntPhys 2, with the best results reaching about 52% accuracy, and finds that all probed features from both vision and language encoders perform at chance level on IntPhys 2. The same study argues that models often hallucinate scene content, omit critical motion details, and fail to exploit physical-plausibility cues reliably (Ballout et al., 22 Jul 2025).

6. Position within intuitive-physics research

IntPhys 2 occupies a specific position within the broader literature on intuitive-physics benchmarks. Relative to the original IntPhys, it is a more realistic, more occlusion-heavy, and more memory-intensive successor. This distinction matters empirically: a 2025 study on self-supervised video pretraining reports that V-JEPA achieves 98% zero-shot accuracy on IntPhys, with strong gains for object permanence, continuity, and shape constancy, while pixel-space video prediction and multimodal LLMs remain near chance on that earlier benchmark (Garrido et al., 17 Feb 2025). In contrast, on IntPhys 2, the benchmark paper reports 57.51% for V-JEPA 2 on the main set, which indicates that IntPhys 2 exposes failure modes not resolved by success on the original benchmark (Bordes et al., 11 Jun 2025).

Related benchmarks push in other directions. Physion++ is designed to test online inference of latent physical properties such as mass, friction, elasticity, and deformability, using an Object Contact Prediction task with an inference phase and a prediction phase. Its authors explicitly contrast it with earlier datasets such as Physion and IntPhys, arguing that those benchmarks typically do not require latent-property inference or only test directly observable properties (Tung et al., 2023). IntPhys 2 does not replace that line of work; rather, it sharpens the evaluation of object persistence, identity, continuity, and solidity under more realistic rendering and occlusion.

Research on model internals further clarifies the benchmark’s role. An occlusion-aware object-centric method from 2020 showed large gains on the original IntPhys, especially on occluded cases, by maintaining latent object tracks through invisibility (Riochet et al., 2020). More recently, probing of video diffusion models has shown that physical plausibility is linearly decodable from intermediate diffusion-transformer states on IntPhys and InfLevel, reaching around 81.27% average accuracy and outperforming dedicated representation-learning baselines such as V-JEPA 2 and VideoMAE-Large on those probing tasks (Esmati et al., 3 Jun 2026). These results do not establish mastery of IntPhys 2 itself, but they suggest that physically meaningful internal structure can exist even when benchmark-level behavior remains weak.

Taken together, the current evidence supports a narrow but important conclusion. IntPhys 2 is not a generic video-classification benchmark; it is a stress test for persistent object representations, short-term memory across occlusion, viewpoint robustness, and physically constrained temporal reasoning. A plausible implication is that the benchmark is useful precisely because it separates basic visual competence from the stronger form of intuitive-physics behavior that humans display almost effortlessly. The published limitations remain substantial: IntPhys 2 is still synthetic, it covers only a limited set of physical principles, and humans retain an evaluation advantage because they can view the full videos while models are typically restricted to subsampled frames and limited context windows (Bordes et al., 11 Jun 2025).