---
title: 'PhysX-Bench: Simulation-Ready 3D Benchmark'
url: https://www.emergentmind.com/topics/physx-bench
type: topic
---

# PhysX-Bench: Simulation-Ready 3D Benchmark

to=arxiv_search  彩神争霸代理  彩神争霸可以json
{"query":"2605.21572 PhysX-Omni PhysX-Bench", "max_results": 5}
to=arxiv_search ӡамjson
{"query":"PhysX-Bench simulation-ready physical 3D generation benchmark", "max_results": 10}
to=arxiv_search  аптономio าคาร่json
{"query":"2505.00337 T2VPhysBench", "max_results": 5}
PhysX-Bench is a benchmark for **simulation-ready physical 3D generation** that evaluates whether a model can generate a 3D object whose **physical properties, part semantics, and simulated behavior are correct and believable in the real world**, rather than merely producing a plausible-looking shape. Introduced together with PhysX-Omni and the PhysXVerse dataset, it is designed for **ground-truth-free evaluation** of physical 3D generation from single images by combining **physics-based simulation** with a large open-source vision-language model (VLM). The benchmark is explicitly aimed at **in-the-wild scenarios**, where full 3D supervision is typically unavailable, and it evaluates both **generative capability** and **understanding capability** through six key attributes: geometry, absolute scale, material, affordance, kinematics, and function/semantic description [2605.21572].

## 1. Origin and benchmark rationale

PhysX-Bench was introduced to address two shortcomings that the paper identifies in existing 3D generation methods and benchmarks. First, prior work is described as focusing too much on **appearance and geometry**, while ignoring the physical attributes required for simulation and robotics. Second, prior evaluation protocols are described as depending on **ground-truth annotations**, which makes them unsuitable for real-world inputs where such supervision is unavailable.

Within that framing, PhysX-Bench is presented as the benchmark component of a broader effort toward **simulation-ready physical 3D assets**. Its purpose is not only to ask whether a generated object is visually convincing, but whether it is physically useful: whether it has plausible scale, material behavior, affordances, articulations, and part-level semantics. The benchmark is therefore meant to enable what the paper calls “robust and realistic evaluation in in-the-wild scenarios” [2605.21572].

A central design choice is that the benchmark is **ground-truth-free in its main evaluation mode**. Instead of comparing a generated asset directly against a fully annotated target object, it evaluates rendered outputs of the generated asset and uses simulation and VLM-based judgment to score the result. This suggests a shift from reconstruction-style benchmarking toward deployment-oriented assessment: the question becomes whether the object behaves plausibly under physically meaningful tests, not merely whether it matches an unavailable reference mesh.

## 2. Scope, inputs, and benchmark construction

PhysX-Bench is constructed around generated outputs rather than around annotated target 3D assets. Conditioning inputs come from both **real-world photographs** and **rendered images of 3D assets**, and the benchmark is described as covering a wide range of common object categories and challenging in-the-wild cases. Evaluation is performed using rendered images or videos of the generated object, rather than raw physical attributes.

The benchmark’s main evaluation pipeline uses the open-source VLM **Qwen3.5-122B-A10B**. The paper motivates this design in explicitly reproducibility-oriented terms:

> “To guarantee the reproducibility and robustness of the benchmark, we adopt the open-source VLM (Qwen3.5-122B-A10B) to evaluate the generated physical attributes. Moreover, to reduce the difficulty of understanding complex 3D structures and physical properties, we use rendered images or videos as inputs for evaluation rather than directly feeding physical attributes.”

This indirect protocol is consequential. It means that PhysX-Bench evaluates an asset through its **visible structure**, its **rendered appearance**, and its **simulated behavior**, rather than through hidden annotations. A plausible implication is that the benchmark is intentionally closer to downstream embodied and simulation settings, where evaluation often proceeds from observable consequences rather than latent labels.

The paper does not specify formal benchmark splits in the main text. It states that PhysX-Bench includes conditioning images from real-world photographs and rendered 3D assets, and that more detailed construction information appears in supplementary material. Accordingly, any stronger claim about train/validation/test partitioning would go beyond what is stated in the main text.

## 3. Evaluated attributes and their operational meaning

PhysX-Bench spans **six key attributes**. The paper also summarizes these as covering “3D structure, appearance, fundamental physical attributes, and understanding.”

| Attribute | What it evaluates | Main evaluation signal |
|---|---|---|
| Geometry | 3D structure and appearance | Rendered images |
| Absolute scale | Correct real-world size | VLM-estimated dimension comparison |
| Material | Plausible physical material properties | Simulation videos |
| Affordance | Plausible human-object interaction priors | VLM common-sense judgment |
| Kinematics | Motion behavior and articulation plausibility | Motion videos |
| Function/semantic description | Part-level semantics and functional meaning | Part-level masks plus reference descriptions |

The benchmark’s **geometry** dimension is not treated as a single scalar notion of mesh quality. Instead, it is decomposed into **CLIP score**, **3D consistency**, and **visual quality**. CLIP score measures alignment between the generated results and the conditioning image; 3D consistency measures structural coherence across multiple rendered views; and visual quality measures perceptual appearance quality using a **reference grading table** with **five levels**, ranging from **very poor** to **excellent**. Geometry in PhysX-Bench therefore includes semantic alignment and perceptual fidelity in addition to multi-view consistency [2605.21572].

**Absolute scale** evaluates whether the generated object has the correct real-world size. The protocol compares the **maximum generated object dimension** with the **VLM-estimated maximum real-world dimension**, then converts the **symmetric percentage error** into a **scale plausibility score**. The paper also notes that, in a conventional supervised setting where ground truth exists, **MSE** can be used for scale.

**Material** is evaluated through dynamic response rather than static appearance alone. The benchmark uses simulation videos under two physical scenarios: **free-fall** and **water-drop**. According to the paper, free-fall behavior, especially at impact, reflects properties like **Young’s modulus** and **Poisson’s ratio**, while water-drop behavior is mainly used to evaluate **density**. This design makes material a behaviorally grounded attribute rather than a texture-recognition problem.

**Affordance** evaluates the plausibility of human-object interaction priors. The paper emphasizes that affordance can have **multiple plausible outcomes**, so the evaluation does not reduce to an exact categorical label. Instead, it considers **local and global plausibility**, **relative ranking plausibility**, **salient misranking of typical parts**, and **overall rationality of predicted affordances**. The benchmark assigns higher scores to predictions that are more consistent with human common sense.

**Kinematics** evaluates whether motion behavior and articulation are physically and semantically plausible. The benchmark uses motion videos and predicted motions inferred from the conditioning image, and defines three components: **prior-part motion consistency**, **revealed-entity plausibility**, and **global articulation coherence**. The paper states that “The final kinematics score is computed as a weighted average of the prior-part motion consistency, revealed-entity plausibility, and global articulation coherence scores.”

**Function description / semantic description** evaluates whether the generated object preserves part-level semantics and functional meaning. The protocol renders **part-level masks** on the generated 3D object and uses the VLM to determine whether masked regions semantically match the **human-annotated reference descriptions** from the conditioning image. In the paper’s tables, this attribute appears as **Description**.

## 4. Evaluation protocol, reproducibility, and human alignment

PhysX-Bench is designed for **automatic, reproducible, and human-aligned evaluation**. Its core methodological claim is that evaluation in wild settings should not depend on unavailable ground truth, yet it should still correlate with human judgment. To validate that claim, the paper measures alignment between automatic evaluation scores and human preference scores using **Spearman’s rank correlation coefficient** [2605.21572].

The reported human-alignment results are strong. The paper gives the following correlations:

- **absolute scale**: $\rho = 1.0$
- **affordance**: $\rho = 1.0$
- **material**: $\rho = 1.0$
- **description**: $\rho = 1.0$
- **kinematics**: $\rho = 1.0$, with Pearson $r = 0.992$
- **geometry**: $\rho = 0.8$, $r = 0.803$

These values are used in the paper to argue that PhysX-Bench rankings are closely aligned with human preferences. The geometry result is somewhat lower than the other attributes, which suggests that perceptual and structural judgments may be harder to compress into a single automated signal than physically grounded dimensions such as scale or material response.

Alongside the main benchmark, the paper also reports conventional supervised metrics for settings where ground truth is available. These include **PSNR** for rendered appearance, **Chamfer Distance (CD)**, **F-score**, **MSE** for absolute scale, **heatmap-based PSNR** for material, affordance, and description, and **MSE** on articulation parameters for kinematics. Generated and ground-truth assets are rendered from **30 viewpoints** and the results are averaged. The benchmark’s main claim, however, is that PhysX-Bench is more general than these metrics because it remains applicable when full ground truth is absent.

A common misconception is that PhysX-Bench is simply another geometry benchmark with a few auxiliary scores. The benchmark design contradicts that interpretation: material and kinematics are assessed through simulation videos, affordance through common-sense ranking plausibility, and description through masked semantic grounding. It is therefore more accurate to regard it as a **hybrid simulation-and-VLM evaluation framework** than as a pure reconstruction benchmark.

## 5. Baselines and reported results

PhysX-Bench is used in the paper to compare PhysX-Omni against four prior methods: **Articulate-Anything**, **MonoArt**, **PhysXGen**, and **PhysX-Anything**. The paper notes that these methods differ in what physical properties they model and in how they represent structure.

The main quantitative table reports the following dimensions: **CLIP**, **3D Consistency**, **Visual Quality**, **Absolute Scale**, **Material**, **Affordance**, **Kinematic**, and **Description**. For **PhysX-Omni**, the reported values are:

| Metric | PhysX-Omni |
|---|---:|
| CLIP | 0.767 |
| 3D Consistency | 64.48 |
| Visual Quality | 90.0 |
| Absolute Scale | 64.26 |
| Material | 59.89 |
| Affordance | 70.57 |
| Kinematic | 80.72 |
| Description | 39.02 |

The paper emphasizes that PhysX-Omni is especially strong on **absolute scale**, **material**, **affordance**, **kinematics**, and **description**. It also notes that **MonoArt** is somewhat stronger on geometry-related metrics because of its reliance on a strong **TRELLIS** geometry pipeline, but that it lacks the same depth of physical reasoning [2605.21572].

This comparison underlies one of the benchmark’s main interpretive points: strong geometry or appearance performance does not imply strong physical understanding. PhysX-Bench is explicitly designed to expose that tradeoff. A plausible implication is that evaluation protocols centered on geometry alone may systematically overestimate readiness for simulation and robotics use.

The benchmark is also described as difficult because it handles **noisy, in-the-wild inputs**, evaluates **implicit physical properties**, and targets attributes that are only partially observable from a single image. In particular, **affordance** and **kinematics** are described as inherently ambiguous and dependent on commonsense reasoning. This makes the benchmark substantially harder than conventional 3D reconstruction evaluation.

## 6. Position within the broader benchmark landscape

PhysX-Bench belongs to a broader family of physics-oriented benchmarks, but its particular contribution lies in focusing on **simulation-ready physical 3D assets** under **ground-truth-free** evaluation. Other recent benchmarks illuminate adjacent but distinct problem formulations.

In text-to-video generation, **T2VPhysBench** evaluates whether generated videos obey **twelve fundamental physical laws** using a **fully manual** human-evaluation protocol. It targets first-principles physical consistency in video generation rather than 3D asset simulation readiness, and it reports that all ten evaluated models score poorly, with all models below **0.60** on average in every law category [2505.00337]. The comparison is instructive: both benchmarks reject purely appearance-centric assessment, but T2VPhysBench operationalizes physical consistency through human judgment over rendered video, whereas PhysX-Bench uses simulation and VLM judgment over generated 3D assets.

In agentic physical reasoning, **Gravity-Bench-v1** is an environment-based benchmark for **physics discovery by agents**. It requires active observation planning under a budget in a **REBOUND** gravitational simulator and evaluates inference of hidden laws and parameters, including out-of-distribution modified gravity and drag cases [2501.18411]. The problem is scientific discovery under partial observability, not 3D generation, but it shares with PhysX-Bench the use of simulation as a testbed for physical reasoning rather than static visual scoring.

**PHYRE** provides a deterministic **2D simulated physics environment** of classical mechanics puzzles with a focus on **sample efficiency** and **cross-puzzle transfer**. Its agents act by placing one or more bodies to satisfy a symbolic goal, and evaluation emphasizes few-attempt success through **AUCCESS** [1908.05656]. Here again, the shared theme is that benchmark value comes from operational interaction with physical dynamics rather than from passive recognition.

For fluid and continuum dynamics, **LagrangeBench** standardizes **SPH-generated particle trajectories** and evaluates learned surrogate models using rollout MSE, **Sinkhorn distance**, and **kinetic energy MSE** [2309.16342], while the benchmark of **incompressible Navier–Stokes equations** provides a large suite of **1044 2D test cases** and **730 3D test cases** for comparing numerical schemes under matched computational budgets [2112.05309]. These benchmarks target solver fidelity rather than object generation, but they reinforce a common methodological pattern: physically meaningful benchmarking requires dynamic evaluation, reproducible protocols, and metrics that extend beyond surface appearance.

Against that backdrop, PhysX-Bench is distinctive in combining **physics-based simulation** and **VLM-based judgment** for **ground-truth-free** evaluation of generated 3D objects from single images. Its emphasis on **scale**, **material response**, **affordance**, **articulation**, and **semantic grounding** places it at the intersection of generative modeling, embodied AI, and simulation-centric evaluation.

## 7. Significance, interpretive cautions, and implications

The immediate significance of PhysX-Bench is methodological. It proposes that evaluation of physical 3D generation should be based on whether assets are **simulation-ready**, not merely whether they look plausible under rendering. The benchmark therefore reframes quality in terms of physical usability for **simulation, robotics, and embodied AI**.

A second significance lies in its treatment of **understanding capability** and **generative capability** as jointly evaluable. The paper explicitly asks not only whether a model can synthesize a correct 3D asset, but also whether it can infer physical attributes from a single image, ground semantics and function in the generated object, and reason about motion and interaction plausibility. This dual role is important because single-image physical generation is underconstrained: many correct outputs require latent inference rather than direct reconstruction.

At the same time, the benchmark’s protocol entails interpretive cautions. Because evaluation is mediated through rendered images or videos and judged by a VLM, benchmark scores are not direct measurements of latent physical parameters. They are measurements of whether physical properties are plausibly recoverable from observable consequences under the benchmark’s protocol. This suggests that PhysX-Bench is best interpreted as a **human-aligned proxy for simulation readiness**, not as a substitute for exhaustive engineering validation.

The paper’s results also support a broader conclusion about current 3D generation systems: appearance and geometry alone are not reliable indicators of physical understanding. PhysX-Bench is designed precisely to expose cases where an asset looks convincing yet fails on scale, material, affordance, articulation, or semantic grounding. In that sense, the benchmark functions both as an evaluation tool and as a diagnostic instrument for model design.

For future research, the benchmark implies that progress in simulation-ready generation will likely require models that represent physical attributes and part semantics more explicitly, especially under in-the-wild single-image conditioning. That implication should be read as an inference from the benchmark design and reported tradeoffs, rather than as a direct empirical claim beyond the paper’s stated results.

Source: https://www.emergentmind.com/topics/physx-bench