PhyWorldBench: Physical Realism Benchmark

• PhyWorldBench is a comprehensive benchmark that assesses video synthesis models' ability to simulate physical laws through diverse categories and prompt types.
• It employs a dual evaluation strategy using human ratings and CAP-based MLLM zero-shot scoring to measure semantic adherence and physical commonsense with metrics like ROC-AUC scores.
• Results reveal that while models reliably handle fundamental physics, they struggle with complex interactions and anti-physics scenarios, highlighting the need for improved prompt engineering and simulation integration.

PhyWorldBench is a comprehensive benchmark specifically created to evaluate the physical realism of text-to-video generation models. The benchmark systematically measures models’ ability to produce videos that consistently adhere to the laws of physics, spanning fundamental principles, composite multi-object interactions, and deliberate “anti-physics” scenarios where prompts explicitly instruct the model to violate real-world constraints. By encompassing a wide inventory of curated prompts and scenarios, PhyWorldBench exposes both the strengths and persistent challenges facing state-of-the-art video synthesis technologies in physical simulation fidelity (Gu et al., 17 Jul 2025).

1. Benchmark Structure and Evaluation Framework

PhyWorldBench is built around a systematic taxonomy of physically grounded phenomena, subdivided for fine-grained analysis of generation quality. Its framework comprises:

• Ten major physics categories (e.g., object motion, energy conservation, forces, deformation, collisions, chemical effects, fluid dynamics, and more), each split into five distinct subcategories (such as sliding, falling, or colliding objects) to capture diverse real-world interactions.
• Seven scenario variants are instantiated for each subcategory, collectively spanning a wide variety of initial conditions or object configurations.
• Three prompt types per scenario:
  • Event Prompts: Concise descriptions focusing on the action or event.
  • Physics-Enhanced Prompts: Descriptions augmented with explicit references to physical mechanisms (e.g., “the ball bounces higher because of its elasticity”).
  • Detailed Narrative Prompts: Rich, multi-sentence descriptions that also include background, object attributes, and finer events.
• Anti-Physics Category: Scenarios where the prompt intentionally contradicts physical laws (e.g., “a rock floats upward after being dropped”). This probes whether a model defaults to learned physical consistency or is capable of producing plausible content when instructed to violate real-world constraints.
• The full benchmark amounts to 1,050 curated cases, covering the entire cross-product of categories, subcategories, scenario variants, and prompt styles.

2. Evaluation Methodology and Metrics

PhyWorldBench employs a two-pronged evaluation strategy: large-scale human annotation and automated zero-shot scoring with Multimodal LLMs (MLLMs).

• Human Evaluation: Human reviewers score each generated video on two binary criteria:
  • Semantic Adherence (SA): Whether the video visually contains the correct objects and depicts the described event.
  • Physical Commonsense (PC): Whether the output is physically plausible given known real-world laws.
  • Each score is assigned as 1 (standard met) or 0 (not met). Final accuracy metrics report the fraction of examples per model (and per scenario type) for which either or both criteria are satisfied.
• Context-Aware Prompt (CAP) MLLM Zero-Shot Evaluation: To facilitate scalable automated assessment, PhyWorldBench introduces a simple yet effective MLLM-based method, CAP, which:
  • Informs the MLLM explicitly that the input video is model-generated (possibly not physically consistent).
  • Requests a detailed free-form video description, then queries for structured analysis of present objects, actions, and physical phenomena.
  • Outputs are then assessed for both SA and PC.
  • CAP achieves ROC-AUC scores of 80.3 (SA) and 75.1 (PC), outperforming vanilla MLLM picture-to-text outputs for physical realism assessment.

3. Analysis of Model Performance and Common Challenges

Twelve leading text-to-video generation models were systematically evaluated, including five proprietary systems (Sora-Turbo, Gen-3, Kling 1.6, Pika 2.0, and Luma) and several prominent open-source frameworks (such as Wanx 2.1 and Hunyuan 720p).

Findings indicate:

• Performance on Fundamental Physics: Most models reliably depict basic motion and simple object interactions, but systematic errors accumulate for phenomena requiring precise conservation laws or temporally consistent multi-object dynamics (e.g., momentum conservation in collisions; energy dissipation in rolling or bouncing objects).
• Composite and Anti-Physics Scenarios: Fidelity declines notably in complex cases. In anti-physics scenarios, models typically revert to realistic outcomes (“physics recovery”), failing to follow prompts’ instructions to defy gravity or execute physically impossible effects. This suggests the models have little flexibility in overriding ingrained real-world priors despite prompt-directed intent.
• Temporal and Interaction Artifacts: Common errors include discontinuous or jarring motion, lack of proper causal interaction between objects, and failures to maintain smooth spatial trajectories.
• Comparison of Prompt Styles: Enriching prompts with explicit physical mechanisms (Physics-Enhanced Prompts) improves adherence to physics more than simply elaborating the event narrative, supporting the use of prompt engineering as a lever for increased physical realism.

Model	Overall SA (%)	Overall PC (%)	Special Notes
Proprietary (Pika 2.0, etc.)	Variable	Variable	Strong on basic motion, weak in anti-physics
Open Source (Wanx 2.1, etc.)	Lower	Lower	Generally better on familiar scenarios

4. Technical Methodology and Representative Formulations

The evaluation logic is anchored by binary criteria:

$SA,\, PC \in \{0, 1\}$

A generated video is successful if it passes semantic (object/event matching) and physical plausibility checks. During automated evaluation, the CAP-based MLLM process can be schematically represented as:

def assess_physics_realism(video, prompt): chain_of_thought = MLLM.describe_video(video) result_SA = MLLM.compare_description_to_prompt(chain_of_thought, prompt) result_PC = MLLM.evaluate_physical_plausibility(chain_of_thought) return result_SA, result_PC # Each is 1 or 0

Statistical reporting is by the fraction of successful cases per criterion, with ROC-AUC used for model assessment in the automated setting.

5. Implications for Model Development and Prompt Engineering

PhyWorldBench results suggest several implications:

• Explicit physical cues in prompts (e.g., referencing collisions, breakage, or acceleration) contribute significantly to models’ ability to generate videos that adhere to physical laws.
• Refinement of training objectives and architectures is required to move models beyond surface-level pattern synthesis toward incorporating physically grounded generative mechanisms. Most current models rely on training data rather than true physical simulation, as revealed by their inability to process anti-physics instructions.
• Context-aware, multi-step evaluation such as offered by CAP is recommended for future benchmarks, as it captures both semantic and physical dimensions efficiently and with high reliability.

6. Broader Impact and Recommendations

PhyWorldBench establishes a new standard for the evaluation of physical realism in generative video models and highlights the persistent gap between visual plausibility and genuine physical correctness. The benchmark encourages the integration of physical priors and explicit simulation components within video synthesis pipelines and demonstrates the significant benefit of targeted prompt engineering. Future research directions suggested include incorporating physics engines, further refining prompt conditioning, and leveraging MLLMs for scalable, context-aware evaluation methodologies (Gu et al., 17 Jul 2025).

By exposing the limitations of current models in both routine and adversarial (“anti-physics”) regimes, PhyWorldBench provides the research community with a rigorous tool for diagnosing shortcomings and tracking progress in the development of models capable of true “physical imagination” and accurate simulation of the observable world.