Principia: Relational Physics Tests for Video Models

Abstract: Evaluating physical reasoning in video models is difficult because absolute motion measurements depend on frame rate, object scale, and camera calibration, all of which are often ambiguous or unavailable in generated video. We propose a different approach. When two objects in the same scene obey the same physical law, their motions must satisfy predictable relationships, and these relationships hold independent of calibration. We introduce Principia, a benchmark that evaluates Newtonian physics through relational consistency between paired objects. Principia spans eight phenomena - gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum, and mass-spring oscillation - across translational, rotational, collisional, and oscillatory dynamics, using real-world scenes recorded under controlled protocols. We also introduce a calibration-independent consistency score that quantifies physical violation directly in image space. Across thousands of generations from six state-of-the-art video generators, no model exceeds 0.42 on Principia despite all scoring around 0.8 on VBench. Vision-LLMs are evaluated on their ability to detect relational physics violations, with the best model achieving only 67% accuracy and most performing near chance level.

• The paper introduces a novel benchmark, Principia, to evaluate the physical fidelity of video models using relational tests that expose discrepancies in adherence to physical laws and laws of motion.
• The benchmark tests eight Newtonian phenomena across 500 real-world scenes, establishing that visual plausibility and physical consistency are decoupled; models achieve high perceptual quality with up to 0.42 consistency scores.
• Increasing model parameters for scaling models do not uniformly improve relational fidelity, indicating no guaranteed physical model. Vikash Vishal gets a remarkably tight evaluation.
• The paper suggests relational invariance scoring could be invaluable in evaluating how video models can properly embody theories of motion and physics

Problem formulation and motivation

“Principia: Relational Physics Tests for Video Models” (2609.04200) addresses a central weakness in current video-model evaluation: visually plausible motion can violate physical laws while remaining difficult to diagnose using conventional perceptual metrics. Existing approaches typically assess subjective plausibility, compare trajectories against a single reference, or require metric quantities such as camera calibration, object scale, velocity, or acceleration. Each strategy is problematic for generated videos, where camera parameters and physical dimensions are often unavailable or unreliable.

Principia instead evaluates intra-scene relational invariants. When two objects are subjected to matched experimental conditions, Newtonian mechanics constrains their motions through equalities, ratios, or orderings that are invariant to camera calibration, image scale, and frame rate. For example, two blocks of different mass sliding on identical inclined planes with the same friction coefficient should arrive simultaneously because their acceleration is mass-independent. An individual generated trajectory may appear plausible, but a discrepancy between the paired trajectories directly exposes a violation.

This formulation is important because it changes the evaluation target from “does the motion look reasonable?” to “does the relation between coupled motions satisfy the relevant physical law?” The distinction is illustrated by a generated clip in which a single block slides plausibly down an incline, whereas a paired-block version violates the mass-independence of gravitational acceleration. The violation is visible only through comparison between objects.

Figure 1: A visually plausible single-object trajectory can conceal a relational violation revealed by comparing paired objects.

The benchmark covers eight Newtonian phenomena: gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum dynamics, and mass-spring oscillation. These span translational, rotational, collisional, and oscillatory dynamics. The authors report more than 500 final scenes, derived from approximately 750 recorded videos through controlled acquisition, filtering, and visual augmentation. The benchmark is therefore intended to test both basic qualitative motion and quantitative relational structure.

Dataset construction and physical invariants

Principia’s data-collection protocol is designed around the sensitivity of relational tests to experimental asymmetries. Small release-time offsets, differences in ramp angle, surface irregularities, lateral impulses, or pendulum amplitudes can produce apparent violations unrelated to model behavior. The authors consequently impose matched geometry, synchronized release, controlled contact surfaces, and minimized external forces. Paired objects are manufactured or selected to share the properties that the relevant law assumes constant.

The resulting real-world scenes are tracked in image space using SAM3. Object trajectories are extracted from segmentation masks, and phenomenon-specific events—such as ground impact, rebound apex, incline arrival, turning points, and oscillation reversals—are detected automatically. The benchmark also uses edited first frames to vary backgrounds, lighting, surface appearance, and scene context without requiring new physical recordings. This increases visual diversity while retaining the underlying physical configuration, although it introduces a dependence on the fidelity of the editing process.

The principal continuous score compares two scalar quantities that should be equal under the tested invariant:

$$S_\phi = 1 - \frac{ \left|\mathcal{F}_\phi(o_1)-\mathcal{F}_\phi(o_2)\right| }{ \left|\mathcal{F}_\phi(o_1)\right|+ \left|\mathcal{F}_\phi(o_2)\right| }.$$

Here, $\phi$ denotes the phenomenon and $\mathcal{F}_\phi$ is the corresponding physics-derived quantity. The score is unit-free and bounded between zero and one. A score of one indicates exact relational agreement; the paper notes that $S_\phi=0.95$ corresponds approximately to a 10% relational asymmetry. Projectile and momentum tests that depend on directional ordering rather than equality are scored as the fraction of scenes satisfying the expected ordering.

The tested relations are physically specific rather than generic plausibility judgments. Gravity compares drop heights with squared fall-time ratios. Restitution compares rebound-height-to-drop-height ratios for identical balls. Friction tests equality of arrival times for different masses. Rotational inertia compares arrival times for matched solid and hollow cylinders. Projectile motion tests whether the ball launched from greater height travels farther. Momentum tests whether post-collision displacement increases with incoming momentum and decreases with target mass. Pendulum dynamics compare periods as a function of string length, while mass-spring scenes compare equilibrium extension against suspended mass.

This design makes the benchmark insensitive to unknown metric scale, but not completely independent of experimental assumptions. It presumes that paired objects genuinely share the relevant geometry and material properties, that tracking is reliable, and that the scene remains sufficiently interpretable for event detection. The benchmark therefore measures relational physical fidelity under controlled macroscopic Newtonian conditions, not unrestricted physical understanding.

Evaluation protocol

The authors evaluate six video generators: Wan2.2-5B, Wan2.2-14B, Cosmos-2.5-2B, Cosmos-2.5-14B, Veo-3.1, and Omni. Each model receives an edited first frame and a phenomenon-specific text prompt. Human experimenters, suspension strings, and release mechanisms are removed from the conditioning image to reduce dependence on apparatus appearance. Multiple random seeds are sampled, and generated clips are filtered using a directional-consistency criterion before the relational score is computed.

The filtering step is consequential. Principia assumes that a generated video at least depicts the requested qualitative event—for example, downward motion for a falling object or oscillation for a pendulum. Videos failing this criterion receive a score of zero rather than being treated as valid but physically inaccurate trajectories. This appropriately penalizes gross failures, but it also means that the reported score combines several failure modes: failure to render the intended event, object identity instability, trajectory deformation, and violation of the specific physical invariant.

The benchmark also evaluates four VLMs—Gemini-3.1-Pro, Gemini-3-Flash, Qwen-32B, and Qwen-4B—on their ability to classify physically consistent and anti-physics videos. These videos are generated in Isaac Sim, allowing the authors to construct controlled counterfactual violations that cannot be recorded naturally. VLMs receive the full clip and a phenomenon-specific PASS/FAIL prompt, and are scored by agreement with the known invariant.

Visual quality and physical fidelity are decoupled

The headline result is a strong separation between perceptual quality and relational physics. All six video generators score approximately 0.8 on VBench, while their Principia scores range only from 0.14 to 0.42. The highest overall score is 0.419 for Wan2.2-14B, followed by 0.409 for Omni and 0.379 for Veo-3.1.

Figure 2: Video generators occupy a high-visual-quality but low-physics-fidelity regime, with distinct failure profiles across phenomena.

This result directly challenges the use of perceptual realism as a proxy for physical consistency. Models near the visual-quality frontier are not systematically more reliable on the tested laws. The implication is methodological: VBench-like evaluations cannot establish that a model represents the causal or dynamical structure required for world simulation. A model can preserve texture, object identity for part of a sequence, and approximate motion appearance while failing the relational constraints that determine whether the motion is physically coherent.

The per-phenomenon profiles further show that failure is not uniform. Omni performs relatively well on friction and rotational inertia, with scores of 0.852 and 0.784, but performs poorly on gravity, restitution, projectile motion, and momentum. Wan2.2-14B reaches 0.709 on friction and 0.729 on inertia, yet scores only 0.017 on momentum and 0.231 on projectile motion. Veo-3.1 performs comparatively well on inertia at 0.669 but obtains 0.000 on momentum. The strongest models therefore exhibit selective competence rather than a general physical representation.

The qualitative samples clarify the nature of these failures. Generated clips may contain plausible local motion but fail to preserve paired-object relationships, hallucinate additional objects, deform objects, cause objects to disappear, or replace continuous dynamics with hovering and abrupt transitions.

Figure 3: Qualitative comparisons show that relational violations frequently coexist with object hallucination, deformation, and temporally inconsistent motion.

Momentum is the most consistently difficult phenomenon. Every generator performs poorly, with scores ranging from 0.000 to 0.074 except for the already low values of the stronger systems. This is consistent with the greater complexity of collision scenarios: the model must maintain object identity, contact timing, pre-collision velocity, collision geometry, and post-collision displacement across multiple interacting bodies. The result suggests that apparent competence in single-object motion does not transfer to multi-object impulse transfer.

Scaling does not provide uniform gains

The paper reports contradictory within-family scaling results. Increasing Wan2.2 from 5B to 14B substantially improves restitution by 0.35, gravity by 0.32, friction by 0.40, inertia by 0.26, projectile motion by 0.22, and pendulum dynamics by 0.17. Its overall score rises from 0.205 to 0.419. However, momentum decreases from 0.067 to 0.017, while spring consistency improves only marginally.

Cosmos scaling is weaker and more inconsistent. Moving from 2B to 14B improves friction, inertia, and pendulum performance, but decreases restitution, gravity, and momentum. The larger Cosmos model reaches only 0.199 overall, compared with 0.148 for the 2B model. More inference compute also does not predict fidelity: Cosmos-2.5-14B requires approximately 74 minutes per video versus 66 minutes for Wan2.2-14B, yet scores 0.22 lower overall.

Figure 4: Parameter scaling produces phenomenon-specific gains and regressions rather than monotonic improvement in physical fidelity.

The implication is not that scale is irrelevant. Wan scaling produces substantial gains on several relational tests. Rather, scale alone does not guarantee the acquisition of a coherent physical model. Different phenomena appear to depend on distinct representational and temporal capabilities, and increasing capacity can alter generation behavior in ways that improve one invariant while degrading another. The data do not establish whether this reflects optimization instability, distributional differences, architectural limitations, or interference among learned motion priors; they establish only that monotonic scaling is insufficient under the evaluated configurations.

VLM detection of physical violations

The VLM results are also weak. The best overall agreement is obtained by Gemini-3-Flash at 0.661, followed by Gemini-3.1-Pro at 0.545, Qwen-4B at 0.541, and Qwen-32B at 0.500. Thus, no model exceeds 0.7 agreement, and the strongest model remains far from reliable detection of relational violations.

Gemini-3-Flash performs particularly well on springs and friction, with agreement scores of 0.895 and 0.840, but its performance is much lower on restitution, inertia, and momentum. Gemini-3.1-Pro is worse overall than Gemini-3-Flash despite being the larger or more capable model in the comparison. Qwen-32B also underperforms Qwen-4B overall.

Figure 5: Increasing VLM scale does not systematically improve agreement with relational-physics ground truth.

The scaling result is especially notable because the task is discriminative rather than generative. VLMs are not required to synthesize a trajectory; they must inspect a paired-object relation and classify it. Yet scaling from Qwen-4B to Qwen-32B reduces overall agreement by 0.04, while scaling from Gemini-3-Flash to Gemini-3.1-Pro reduces it by 0.12. Friction exhibits the largest regression in both families. These results indicate that the difficulty is not simply a lack of visual detail or language-model capacity. The models may recognize individual actions and objects while failing to compare quantitative temporal or spatial relations across objects.

The use of Isaac Sim anti-physics videos strengthens experimental control by supplying exact binary ground truth. At the same time, it creates a domain gap between simulated violations and real-world recordings. The VLM results therefore demonstrate poor detection on this controlled testbed, but they do not alone determine whether the same failure rates hold for arbitrary natural videos.

Limitations and open questions

Principia is intentionally restricted to macroscopic Newtonian mechanics. It does not evaluate fluid dynamics, soft-body deformation, thermodynamics, complex contact mechanics, or materially heterogeneous scenes. The benchmark also assumes static or near-static cameras. The authors’ camera-sensitivity analysis shows that most phenomena are robust to moderate synthetic pan and zoom, but projectile scoring is substantially affected by horizontal panning because it uses a binary range-ordering criterion. A 10% horizontal pan reduces the projectile score from 1.000 to 0.417 in the reported analysis.

The benchmark does not automatically distinguish a genuine law violation from failures of basic video generation. Hallucinated objects, disappearing objects, severe deformation, incorrect segmentation, and nonconforming motion can all lead to low scores. The authors explicitly scope Principia to interpretable videos, but the directional filter and zero-score treatment mean that the aggregate metric still reflects both physical and representational failures.

There are also internal methodological assumptions that merit attention. Mass is inferred from cube side lengths under constant-density assumptions in the spring experiment. Momentum is evaluated through displacement ordering rather than a full impulse or energy analysis. The benchmark’s relational invariants are valid only when the apparatus satisfies the stipulated matching conditions, so data quality and calibration of the physical setup remain critical even though the final score does not require camera calibration. Finally, the use of edited first frames increases visual diversity but may produce conditioning images whose appearance is not fully independent of the original recording distribution.

The central open question is therefore specific: whether models trained with explicit relational objectives, differentiable physics constraints, simulator-generated counterfactuals, or architecture-level object-centric dynamics can improve Principia scores without sacrificing visual quality or generalization to unseen apparatus and contexts.

Conclusion

Principia provides a calibration-independent benchmark for testing whether video generators preserve physical relations between paired objects. Across eight Newtonian phenomena, the evaluated generators achieve high perceptual quality but low relational consistency: the best overall score is 0.419, despite VBench scores near 0.8. Performance is highly phenomenon-dependent, momentum is particularly difficult, and parameter scaling produces both substantial gains and clear regressions. VLMs likewise show limited ability to detect relational violations, with the best overall agreement reaching only 0.661.

The paper’s principal contribution is evaluative rather than architectural. It demonstrates that realistic video appearance and physical fidelity are separable properties, and that paired-object invariants provide a quantitative way to expose this separation without requiring metric calibration. For claims that video models function as physical world models, Principia establishes a stringent test: preserving the appearance of motion is insufficient unless the model also preserves the relations imposed by the underlying dynamics.