PhysX-Anything Framework

• PhysX-Anything is a simulation framework that integrates universal PDE surrogate modeling with generation of physically grounded, articulated 3D assets.
• It employs discrete tokenization alongside autoregressive and diffusion transformer architectures to efficiently predict long-horizon physical dynamics and reconstruct detailed geometries.
• Experimental results demonstrate state-of-the-art performance in error reduction, geometry accuracy, and seamless integration into robotics simulation pipelines.

The PhysX-Anything Framework encompasses a set of innovations targeting general-purpose, simulation-ready physical modeling across both the simulation of physical systems and articulated 3D asset generation. The term covers distinct but related advances: (1) the PhysX-Anything vision—a universal approach to physics simulation inspired by the PhysiX foundation model, responsible for scaling up multitask PDE surrogate modeling with discrete tokenization, generative sequences, and refinement; and (2) the PhysX-Anything model for creating articulated, physically grounded 3D assets from single RGB images, which enables direct deployment in simulation environments for robotics, embodied AI, and virtual interaction. Both paradigms share the goal of plug-and-play, cross-domain model generality, realized through novel tokenization, scalable generative architectures, and integration of physical constraints (Nguyen et al., 21 Jun 2025, Cao et al., 17 Nov 2025).

1. Foundations and Scope

The PhysiX foundation model introduces key architectural and methodological principles central to the PhysX-Anything vision in physics simulation (Nguyen et al., 21 Jun 2025). Its primary advances include:

• Discrete tokenization of continuous, multi-channel physical fields, compressing heterogeneous spatiotemporal data into a latent sequence amenable to large-scale generative modeling.
• An autoregressive transformer backbone trained on discretized histories for long-horizon PDE sequence prediction.
• A learned refinement network to correct quantization artifacts, ensuring high-fidelity output suitable for downstream scientific workflows.

Concurrently, PhysX-Anything (Cao et al., 17 Nov 2025) extends these ideas to the domain of simulation-ready physical 3D asset generation, focusing on the problem of producing articulated, simulation-compatible object representations from a single image. Its core components address geometry compression, articulation reasoning, and physical parameter estimation, enabling seamless integration into environments such as MuJoCo.

2. Model Architectures and Tokenization Strategies

Both frameworks rely on discrete representations to facilitate generative modeling at scale:

• PhysiX Tokenization: The model's encoder $E$ maps a continuous field $x \in \mathbb{R}^{H\times W\times C}$ to a compressed, d-channel latent $s=E(x) \in \mathbb{R}^{h\times w\times d}$, then a quantizer $q$ partitions each scalar into one of $K$ bins per channel using a learned codebook. Discrete tokens $z\in\{1,...,K\}^L$, with $L=(H\cdot W)/8^2$, represent each latent frame. This universal token space accommodates cross-task physical dynamics, supporting variable resolution and channel sets (Nguyen et al., 21 Jun 2025).
• PhysX-Anything 3D Tokenization: The system voxelizes meshes into $32^3$ binary occupancy grids and linearizes to index sets $\mathcal{I}$, then merges contiguous indices into index–range tokens, reducing token count by $193\times$ compared to mesh tokenizations (empirically, $x \in \mathbb{R}^{H\times W\times C}$0). This enables explicit geometry learning within standard VLM token budgets, sidestepping the need for custom tokenizers or architectural changes (Cao et al., 17 Nov 2025).
• Autoregressive and Diffusion Backbones: PhysiX employs a 4.5B parameter decoder-only transformer with rotary position encodings; PhysX-Anything uses Qwen2.5 in a similar decoder-only setup conditioned on visual tokens and prompts. Subsequent refinement of geometry is performed by a Controllable Flow Transformer trained under a noise-prediction loss, followed by structured latent diffusion for mesh reconstruction.

3. Physical Parameter and Articulation Modeling

PhysX-Anything advances physical object modeling workflows by tightly coupling geometry, articulation, and physical properties:

• Overall Physical Representation (OPR): The VLM generator outputs a JSON-style OPR encoding global attributes (absolute scale, density, material, friction), a kinematic tree $x \in \mathbb{R}^{H\times W\times C}$1, joint axes, and motion limits $x \in \mathbb{R}^{H\times W\times C}$2. All kinematic and geometric data share a voxel coordinate frame, ensuring consistency.
• Physical Estimation: Physical properties (e.g., mass from predicted density and volume) and articulation descriptors (joint type, axis, origin, motion limits) are inferred and included with each asset. Part segmentation, affordance labeling, and property estimation are integral to the pipeline.
• Simulation Integration: The exported assets are compatible with standard simulation environments via URDF/XML, providing grounded assets for robot learning, physics-based animation, and digital twin construction.

4. Training Methodology and Data Resources

• PhysiX Training Protocol: The model is pre-trained on natural videos—using the Cosmos checkpoint for strong spatiotemporal priors—and fine-tuned on eight canonical 2D PDE datasets sourced from The Well benchmark, with uniform task sampling and dynamic RoPE frequency truncation. Joint training of tokenizer and autoregressive (AR) core leverages cross-task synergies, aided by per-dataset decoders and refinement heads (Nguyen et al., 21 Jun 2025).
• PhysX-Anything and the PhysX-Mobility Dataset: PhysX-Anything is trained on PhysX-Mobility, which expands existing physical 3D datasets to 47 categories and over 2,100 assets, each richly annotated with scale, material, friction, density, part-level affordances, and detailed kinematic trees. This corpus underpins VLM fine-tuning and downstream benchmarking, enabling generalization to both curated and in-the-wild images (Cao et al., 17 Nov 2025).
• No Special Token Requirement: The merged-range tokenization allows use of standard VLM tokenizers, avoiding increased token budgets or ad hoc architectural modifications.

5. Experimental Results and Benchmark Comparisons

Quantitative evaluation demonstrates strong state-of-the-art performance for both frameworks:

PhysiX (The Well Benchmark; Δt=1):

• Achieves lowest average VRMSE in 5/8 tasks and mean rank 1.62 vs. baselines (FNO, TFNO, U-Net, C-U-Net).
• On long-horizon rollouts (Δt up to 56), establishes new state-of-the-art on 18/21 points, reducing errors up to 97% and maintaining structural fidelity in vortex and reaction-diffusion phenomena.
• Ablation shows 30–50% lower errors with video pre-training, superiority of large-scale transformer (4B vs. 2B, 700M), and quantization noise halving via refinement.

PhysX-Anything (PhysX-Mobility and In-the-Wild Images):

Metric	URDFormer	Articulate-Anything	PhysXGen	PhysX-Anything
PSNR (↑)	7.97	16.90	20.33	20.35
Chamfer Dist. (↓)	48.44	17.01	14.55	14.43
F-score @1% (↑)	43.81	67.35	76.30	77.50
Abs. Scale Error (↓)	—	—	43.44	0.30
Material Acc. (↑)	—	—	6.29	17.52
Kin. Params (↑)	0.31	0.65	0.71	0.83

In-the-wild generation achieves VLM metrics of 0.94 (geometry, kinematics) versus previous bests of 0.65/0.61, and human raters award normalized geometry scores of 0.98 for PhysX-Anything. Policy learning with MuJoCo using these assets leads to success rates over 85% within 100k PPO timesteps.

6. Limitations and Trajectories Toward Full Generality

While the current frameworks exhibit strong empirical performance, further innovation is required to realize the fully universal "PhysX-Anything" vision (Nguyen et al., 21 Jun 2025):

• Zero-shot Generalization: Current PhysiX requires fine-tuning for unseen PDEs. Incorporation of meta-learning or contrastive physical pretext tasks is necessary for adapting to new systems or boundary conditions without gradient updates.
• End-to-End Differentiable Tokenization: Joint training of tokenization and AR components would allow physics-informed objectives, such as enforcing conservation laws, to propagate through quantization, reducing discretization artifacts.
• Engine Integration and Real-Time Inference: Extensions are needed for streaming inference and seamless coupling with existing physics engines (e.g., via TensorRT, ML Accelerators), including bidirectional interfacing to guarantee conservation law satisfaction.
• Higher-Dimensional and Irregular Geometries: To support true simulation generality, especially for 3D fluid flows and meshes, further work on tokenization (graph-based, octree) is essential.

7. Impact and Applications

The PhysX-Anything Framework introduces a paradigm shift in the generation and simulation of physically plausible assets. By enabling:

• Plug-and-play simulation of arbitrary PDE systems and real-world-inspired physical assets.
• Compression strategies compatible with large-scale language and vision models.
• Articulated, physically grounded 3D asset creation from a single image, ready for ingestion by standard robotics simulation pipelines.

Applications span scientific computing, robotics policy learning, virtual reality content creation, and digital twins. The ability to rapidly synthesize and simulate complex physical systems and articulated assets bridges long-standing gaps between perception, generative modeling, and downstream physics-based decision-making (Nguyen et al., 21 Jun 2025, Cao et al., 17 Nov 2025).