PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects

Abstract: Simulation-ready physical 3D assets have emerged as a promising direction owing to their broad applicability in downstream tasks. However, most existing 3D generation methods either neglect physical properties or are limited to a single asset category, e.g., rigid, deformable, or articulated objects. To address these limitations, we introduce PhysX-Omni, a unified framework for simulation-ready physical 3D generation across diverse asset types. Specifically, we develop a novel and efficient geometry representation tailored for Vision-LLMs, which directly encodes high-resolution 3D structures without compression, significantly improving generation performance. In addition, we construct the first general simulation-ready 3D dataset, PhysXVerse, covering diverse indoor and outdoor categories. Furthermore, to comprehensively and flexibly evaluate both generative and understanding capabilities in the wild, we propose PhysX-Bench, which encompasses six key attributes: geometry, absolute scale, material, affordance, kinematics, and function description. Extensive experiments with conventional metrics and PhysX-Bench show that PhysX-Omni performs strongly in both generation and understanding. Moreover, additional studies further validate the potential of PhysX-Omni for applications in simulation-ready scene generation and robotic policy learning. We believe PhysX-Omni can significantly advance a wide range of downstream applications, particularly in embodied AI and physics-based simulation.

• The paper introduces a unified VLM-based pipeline for simulation-ready generation of rigid, deformable, and articulated 3D assets.
• It leverages a coarse-to-fine, text-driven geometry encoding and the extensive PhysXVerse dataset to enhance physical attribute prediction.
• Benchmarking via PhysX-Bench demonstrates superior geometric fidelity and physical realism, advancing robotics and embodied AI simulations.

Unified Simulation-Ready Physical 3D Generation: An Analysis of PhysX-Omni

Introduction and Context

PhysX-Omni (2605.21572) proposes a comprehensive VLM-driven framework for simulation-ready 3D asset generation, explicitly targeting the triad of rigid, deformable, and articulated objects. Addressing persistent deficiencies in prior art—limited physical attribute modeling, lack of dataset diversity, and weak evaluation protocols—the authors contribute not only an integrated generative pipeline, but also a richly annotated asset corpus (PhysXVerse) and a multifaceted benchmark suite (PhysX-Bench). The approach demonstrates substantial progress toward reliable 3D asset generation with physical realism and generalization capacity, which is critical for embodied AI, robotics, and interactive simulation domains.

Architecture and Generative Paradigm

PhysX-Omni leverages a coarse-to-fine, global-to-local reasoning mechanism grounded in the capabilities of vision-LLMs. Given a (possibly occluded) input image, the system first infers overall semantic, geometric, and physical parameters at the global object level, forming a tree-structured latent representation suitable for autoregressive language modeling. Subsequent multistage, part-level generation produces explicit parametric geometry alongside critical physical properties such as scale, material, affordance, and articulation.

Figure 1: High-level generative protocol—global inference conditions downstream partwise geometry/physics synthesis yielding directly simulatable outcomes.

Distinctively, PhysX-Omni introduces a tailored text-based geometry encoding: per-part voxelization is combined with run-length encoded 2D slices along the z-axis, further compressed via template-based sharing of redundant layers. This approach minimizes sequence length and redundancy, eschews special vocabulary or tokens, and harmonizes with existing LLM tokenization, in contrast to prior mesh- or VQ-based alternatives.

Dataset Construction: PhysXVerse

PhysXVerse is curated as the first simulation-focused, general-purpose 3D dataset, comprising over 8.7K assets covering 2.9K+ categories from indoor, outdoor, robotic, and vehicular domains. Sourced from PartVerse, assets are processed with human verification for geometry and a human-in-the-loop, VLM-assisted annotation for physical properties.

Figure 2: PhysXVerse category, part count, and semantic diversity distribution compared to existing datasets.

The result is a dataset with significant coverage of complex, articulated, and deformable objects, long-tailed part distributions, and diverse physical annotations, enabling robust training and assessment of generalized sim-ready generation methods.

Benchmarking: PhysX-Bench

PhysX-Bench is developed to evaluate both the generative and understanding capabilities of 3D models without reliance on ground-truth 3D/physical labels. It spans six axes: geometry, scale, material, affordance, kinematics, and semantic description, using VLM-driven scoring on rendered images and simulation videos. Notably, physical property assessment includes visual dynamics in free-fall and water-drop scenarios, allowing indirect inference of elasticity, density, and part affordance in a manner empirically validated to align with human judgments.

Figure 3: PhysX-Bench architecture—multidimensional evaluation encompassing structure, physics, and semantics.

Experimental Evaluation and Comparative Analysis

PhysX-Omni is trained on a large-scale asset corpus (PhysXNet, PhysX-Mobility, and PhysXVerse), using multi-view conditioning to reinforce appearance/structure correspondences. Quantitative evaluation on PhysXVerse and PhysX-Mobility—benchmarked against state-of-the-art methods such as PhysXGen, PhysX-Anything, MonoArt, and Articulate-Anything—demonstrates the following:

• Geometric Fidelity: PhysX-Omni achieves superior PSNR, CD, and F-score, with marked improvements for articulated/complex objects.
• Physical Attribute Prediction: Large gains in absolute scale error (reduced to ∼2.8), material, affordance, and kinematics (kinematic score >0.91 on PhysXVerse), indicating more accurate predictive modeling of real-world properties.
• Generalization: Strong performance across both synthetic and real in-the-wild samples, with balanced scores on all PhysX-Bench metrics.

  Figure 4: Quantitative comparison—PhysX-Omni outperforms prior art across geometric and physical evaluation; high human alignment correlations for PhysX-Bench scores.

Qualitative results substantiate these findings: PhysX-Omni yields detailed, coherent multi-part assets with plausible deformation, articulation, and material-consistent simulation behavior.

Figure 5: Visual quality and attribute richness of assets generated by PhysX-Omni compared to baselines.

Figure 6: Assets generated with deformability, exhibiting realistic free-fall deformation dynamics in simulation.

Figure 7: Visualization confirms superior complex structure generation with PhysX-Omni’s geometry representation versus baselines.

Ablation and Human Alignment

Ablative comparisons reveal the explicit, template-based text geometry encoding brings substantial improvements over naive voxel-index or segmentation-dependent schemes, notably for part continuity and kinematic stability.

Human evaluation establishes a Spearman correlation of $\rho = 1.0$ for physical attribute metrics and $\rho = 0.8$ for geometry, supporting the credibility of the automated evaluation protocol.

Applications and Practical Deployment

PhysX-Omni assets are directly imported into physics simulators for robotic policy learning, preserving physical realism and articulation fidelity without manual post-processing.

Figure 8: Demonstration of physically plausible, simulator-ready asset manipulations in robotic tasks using PhysX-Omni outputs.

The framework also enables rapid generation of full sim-ready environments, integrating image-driven depth and segmentation with asset synthesis for scene construction.

Figure 9: Automated sim-ready scene assembly using PhysX-Omni—enabling application in embodied AI and physics-driven world generation.

Implications and Future Directions

PhysX-Omni concretely advances the state-of-the-art in unified 3D asset generation with explicit physical simulation compatibility, reliable evaluation, and real-world deployability. Strong results on scale estimation, kinematics, and partwise geometry raise the standard for embodied AI simulation preparation and robotics.

There remain limitations in the handling of highly complex or visually challenging geometries, which the authors attribute to the framework’s current emphasis on physical realism over purely appearance-driven metrics. Extensions may include integration of denser appearance priors, scaling with larger geometry corpora, and tighter synergy between voxel-based and mesh-based decoders for photometric fidelity.

Conclusion

PhysX-Omni (2605.21572) represents a significant consolidation of simulation-ready asset generation, combining advances in geometry encoding, dataset curation, and holistic benchmark design. The demonstrated improvements in both structural and physical attribute domains, alongside validated practical deployment capability, provide a new baseline for research and applications in robot simulation, embodied AI, and scalable world modeling.