PhysXVerse: Unified 3D Asset Dataset

• PhysXVerse is a curated asset-level dataset of simulation-ready 3D objects with diverse physical annotations for scale, material, affordance, and kinematics.
• It offers over 8.7K high-quality assets across 2.9K+ categories, covering simple rigid structures to complex articulated systems.
• The dataset employs a human-in-the-loop annotation pipeline and integrates with PhysX-Omni to enable robust simulation and high-resolution physical generation.

PhysXVerse is a curated asset-level corpus of simulation-ready physical 3D objects introduced alongside "PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects" (Cao et al., 20 May 2026). In that work, it is defined as “the first general simulation-ready physical 3D dataset,” intended to provide unified supervision for rigid, deformable, and articulated object generation. Its distinguishing role is not only dataset scale, but the combination of broad category diversity, simulator-oriented part structure, and physical annotations such as absolute scale, affordance, material, functional description, and kinematic information. Within the PhysX-Omni framework, PhysXVerse serves simultaneously as a training source, an evaluation dataset, and the data substrate that makes unified physical 3D generation technically plausible (Cao et al., 20 May 2026).

1. Definition, scope, and reported scale

PhysXVerse is presented as a dataset of simulation-ready 3D assets, not as a conventional scene dataset. The paper first describes it as containing “over 8K assets spanning more than 2K indoor and outdoor categories,” and later gives the more precise count “more than 8.7K high-quality simulation-ready 3D assets spanning over 2.9K categories” (Cao et al., 20 May 2026). The named coverage includes indoor furniture, unmanned aerial vehicles, robots, vehicles, and large-scale scene components; examples include helicopters, tanks, racing cars, skyscrapers, toys, cars, buildings, human models, and robots.

Reported characteristic	Value
Asset count	More than 8.7K
Category count	Over 2.9K
Part-count range	1 to 65

A further structural statistic is central to the dataset’s claim of generality: “The number of parts ranges from 1 to 65,” and the part-count distribution is described as long-tailed (Cao et al., 20 May 2026). At the low end, this covers simple rigid structures; at the high end, highly complex articulated systems. This suggests that PhysXVerse is designed to span multiple object regimes within a single representational and annotation framework, rather than treating rigid, deformable, and articulated assets as separate datasets.

The paper is also explicit about what PhysXVerse is not. It does not provide train/validation/test splits in the provided text, does not report per-category counts, and does not give an explicit rigid/deformable/articulated breakdown. It also does not document a scenes-versus-objects split in the conventional dataset-card sense. Because training is described by rendering “25 images for each object from different viewpoints,” and because the collection is repeatedly referred to as a dataset of “3D assets,” the most defensible characterization is that PhysXVerse is primarily an asset-level dataset rather than a scene-level one (Cao et al., 20 May 2026).

2. Source corpus and annotation pipeline

PhysXVerse is constructed from PartVerse and uses PartVerse’s “human-verified segmentation annotations” as its structural basis (Cao et al., 20 May 2026). The dataset is therefore neither described as being reconstructed from raw sensor scans nor generated entirely from scratch within the PhysX-Omni pipeline; instead, it is curated, filtered, and augmented from an existing 3D asset repository.

The reported construction pipeline proceeds in a staged human-in-the-loop manner. First, the original source is preprocessed by “filtering invalid samples and merging excessively small or noisy parts to improve structural consistency.” Second, the authors render multi-view images of each 3D asset. Third, they employ “a powerful VLM (GPT)” to generate preliminary physical annotations, specifically including absolute scale, affordance, material, functional descriptions, and kinematic information. Fourth, those annotations are “verified and refined by human annotators to ensure both physical plausibility and annotation quality” (Cao et al., 20 May 2026).

This pipeline matters because PhysXVerse is explicitly framed as a response to the scarcity of large-scale, high-quality annotated 3D datasets for physical generation. The VLM-assisted step provides breadth across more than 2.9K categories, while the human verification step is the mechanism used to preserve physical plausibility. The paper also states that it adopts a human-in-the-loop annotation pipeline from prior work, reinforcing that the dataset is neither purely manual nor purely automatic in its labeling regime (Cao et al., 20 May 2026).

A plausible implication is that PhysXVerse is designed to optimize annotation scalability under simulator-oriented constraints: broad taxonomic coverage is retained, but structural consistency is enforced early so that downstream part-based geometry learning remains feasible.

3. Simulation readiness and annotation semantics

In the PhysX-Omni paper, “simulation-ready” is not treated as a vague label. PhysXVerse is simulation-ready because it couples usable part-structured geometry with annotations intended to support physically grounded simulation and reasoning (Cao et al., 20 May 2026). The explicitly named annotation types are:

• Absolute scale
• Affordance
• Material
• Functional descriptions
• Kinematic information

These fields align with the outputs that later appear in PhysX-Omni generation and evaluation. The paper repeatedly ties simulation readiness to geometric structures, physical properties, and articulated parameters, and in the robotics section it states that generated assets are imported into a simulator together with “their geometric structures, physical properties, and articulated parameters” (Cao et al., 20 May 2026).

The annotation regime appears to vary by object type, although the paper does not provide separate formal schemas. For rigid objects, the relevant fields are geometry, absolute scale, material, affordance, and description. For articulated objects, kinematic information is essential, and later evaluation refers to articulation parameters such as joint axis positions, joint directions, joint types, and motion limits. For deformable objects, material evaluation is linked to behavior in free-fall and water-drop scenarios, where contact and fluid response are interpreted through properties such as Young’s modulus, Poisson’s ratio, and density. However, the paper does not explicitly state that PhysXVerse stores numeric Young’s modulus, Poisson’s ratio, or density for every asset (Cao et al., 20 May 2026).

That omission is important. The safest reading is that PhysXVerse contains material annotations intended to encode simulator-relevant physical behavior, but the exact per-asset parameterization is not specified in the provided text. Likewise, the paper does not provide a public storage schema or file-format specification for the dataset itself. No URDF, USD, OBJ, GLTF, or simulator-native serialization format is named for PhysXVerse in the provided material (Cao et al., 20 May 2026).

4. Preparation for learning and relation to PhysX-Omni

Although PhysXVerse is a dataset contribution, its design is closely coupled to the PhysX-Omni learning stack. The paper states that the authors “first voxelize the simulation-ready assets and decompose them into part-level voxels according to the annotated object structure” (Cao et al., 20 May 2026). Each part voxel is then sliced along the $z$-axis into 2D binary masks and encoded with a template-based run-length encoding. This representation is described as a novel geometry representation tailored for vision-LLMs, and it is used as the training target for high-resolution physical 3D generation.

The important distinction is that this voxel-and-slice representation is a learning representation, not necessarily the native storage format of PhysXVerse. The dataset is curated so that such a representation is possible: clean part segmentation, merged noisy fragments, and structural consistency are prerequisites for partwise voxelization and downstream tokenization. The paper also refers to a “tree-structured and VLM-friendly formulation” for global asset representation, following the PhysX-3D / PhysXGen line, again indicating that PhysXVerse is deliberately aligned with model-side structural priors rather than being a raw mesh repository (Cao et al., 20 May 2026).

PhysXVerse is one of three datasets used for training PhysX-Omni. The paper states that combining PhysXNet, PhysX-Mobility, and PhysXVerse yields “more than 42K simulation-ready physical 3D assets spanning diverse indoor and outdoor categories,” covering rigid, articulated, and deformable objects with rich geometric structures and physical attributes (Cao et al., 20 May 2026). Within that combined corpus, PhysXVerse is the component emphasized for its diversity. The teaser caption makes this explicit: “By exploiting the high diversity of PhysXVerse, PhysX-Omni is capable of generating detailed and general 3D assets covering rigid, deformable, and articulated objects.”

This suggests that PhysXVerse’s central function is to provide a common substrate across heterogeneous physical regimes. Instead of training separate models or type-specific heads on isolated corpora, PhysX-Omni learns from a dataset in which part structure, scale, material, affordance, description, and kinematics are aligned wherever relevant.

5. Evaluation role and quantitative evidence

PhysXVerse is not only a training source; it is also one of the principal evaluation datasets in the paper (Cao et al., 20 May 2026). The conventional evaluation includes geometry, absolute scale, material, affordance, kinematics, and description-related criteria. The paper defines some of these metrics explicitly: for absolute scale, it computes the Mean Squared Error (MSE) between predicted and ground-truth object scales; for kinematics, it computes the MSE between predicted and ground-truth articulation parameters, including joint axis positions, joint directions, joint types, and motion limits.

On PhysXVerse itself, PhysX-Omni reports the following results:

• PSNR: 21.52
• Chamfer Distance: 2.95
• F-score: 91.28
• Absolute scale error: 2.79
• Material: 27.23
• Affordance: 21.47
• Kinematic: 0.9185
• Description: 31.05

The paper further notes that Chamfer Distance is reported in units of $\times 10^{-3}$, and F-score in units of $\times 10^{-2}$ under a distance threshold of 0.05 (Cao et al., 20 May 2026). The most dramatic comparison is in absolute scale: PhysX-Omni reduces the error from 309.31 for PhysXGen and 298.19 for PhysX-Anything to 2.79 on PhysXVerse. For kinematics, the score rises from 0.4191 for PhysX-Anything and 0.3805 for MonoArt to 0.9185.

The paper does not include a controlled ablation isolating “with PhysXVerse” versus “without PhysXVerse.” Accordingly, any claim about the dataset’s causal contribution to those gains must remain interpretive. Still, the empirical role of PhysXVerse is unusually direct: it is part of the training corpus, one of the main evaluation datasets, and one of the paper’s qualitative demonstrations of diversity (Cao et al., 20 May 2026).

For human-alignment validation of the broader evaluation protocol, the paper reports exact correlation values including Spearman $\rho=1.0$, Pearson $r=0.992$ for kinematics, and geometry correlation values $\rho=0.8$, $r=0.803$ (Cao et al., 20 May 2026). These figures are not dataset-construction statistics, but they matter because PhysXVerse is used within the same evaluative framework for physical plausibility and understanding.

6. Relation to PhysX-Bench and adjacent research

PhysXVerse is conceptually paired with PhysX-Bench, which the paper introduces as “the first physical 3D generative benchmark” for ground-truth-free evaluation in real-world and in-the-wild settings (Cao et al., 20 May 2026). PhysX-Bench evaluates six attributes: geometry, absolute scale, material, affordance, kinematics, and description. These six axes mirror the annotation targets of PhysXVerse. In that sense, PhysXVerse supplies the supervision, while PhysX-Bench operationalizes the same physical and semantic dimensions at evaluation time.

The dataset also occupies a distinct niche within a broader movement toward simulation-ready representations. PIXIEVERSE, for example, is described as “one of the largest known datasets of paired 3D assets and physic material annotations,” but it contains 1624 high-quality single-object assets spanning 10 semantic classes and is oriented toward supervised physics-from-pixels and dense material-field prediction rather than unified rigid/deformable/articulated 3D asset generation (Le et al., 20 Aug 2025). PhysiX, by contrast, is a 4.5B-parameter autoregressive foundation model for 2D multichannel field simulations on The Well benchmark; it addresses heterogeneous PDE dynamics rather than asset-level 3D physical object corpora (Nguyen et al., 21 Jun 2025). GS-Verse studies mesh-based Gaussian Splatting for physics-aware interaction in virtual reality and focuses on a mesh-guided rendering-and-simulation coupling layer, not on a broad annotated asset dataset (Pechko et al., 13 Oct 2025).

These comparisons clarify PhysXVerse’s specific contribution. It is neither a dense material-field corpus for static inference nor a simulator-side mechanics benchmark nor a PDE trajectory dataset. Its domain is simulation-ready 3D assets with part structure and physically meaningful annotations broad enough to support unified generation, understanding, scene assembly, and robotic-policy-oriented simulation pipelines (Cao et al., 20 May 2026).

7. Documentation gaps, limitations, and open questions

The most conspicuous limitations are documentary rather than conceptual. The provided text does not specify dataset splits, per-category frequencies, per-type counts, or explicit scene/object partitions (Cao et al., 20 May 2026). It also does not provide a formal released schema for annotation storage, does not enumerate simulator-native file formats, and does not explicitly state whether every asset includes numeric constitutive parameters beyond categorical or descriptive material annotations.

The paper’s explicit limitations section is written primarily about PhysX-Omni rather than PhysXVerse, but it has implications for the dataset regime. The authors state that “geometric quality can still be improved for highly complex structures and fine-grained details,” and that the framework may underperform on certain appearance-focused geometric metrics because it emphasizes unified physical understanding and simulation-ready generation rather than appearance-oriented geometry pre-training (Cao et al., 20 May 2026). A plausible implication is that, even with PhysXVerse, the current data-and-representation pipeline leaves room for improvement in high-frequency geometric fidelity.

Another unresolved issue concerns annotation reliability at scale. The dataset uses GPT-assisted preliminary labels followed by human verification, which indicates an explicit attempt to balance breadth and quality. At the same time, the need for refinement implies that raw VLM-generated physical annotations are not treated as sufficient on their own (Cao et al., 20 May 2026). The paper does not quantify annotation noise, inter-annotator agreement, or simulator-side failure rates for specific annotation types.

Taken together, these limitations define PhysXVerse’s present status clearly. It is an asset-level, part-structured, physically annotated corpus built from PartVerse, containing more than 8.7K assets across more than 2.9K categories, with part counts from 1 to 65 and annotation targets spanning scale, material, affordance, description, and kinematics (Cao et al., 20 May 2026). Its significance lies in enabling a unified treatment of rigid, deformable, and articulated 3D generation. Its remaining open questions concern dataset documentation granularity, released schema details, and the extent to which richer physical parameterizations can be made explicit in future versions.