PhysGaussian: Unified 3D Simulation & Rendering

Updated 11 May 2026

PhysGaussian is a framework that integrates physically-informed simulation with 3D Gaussian primitives, enabling unified rendering and dynamic scene synthesis.
It employs a Material Point Method coupled with explicit Gaussian kernels, eliminating the decoupling of geometry and simulation while ensuring high fidelity.
Applications span real-time generative view synthesis, uncertainty-aware physical reasoning, and adaptive modeling for diverse materials and dynamic scenes.

PhysGaussian refers to a family of frameworks and representations that directly integrate physically grounded simulation with 3D Gaussian scene primitives, enabling unified approaches to generative dynamics, physically realistic rendering, dynamic scene synthesis, and physical property inference. Across methods, PhysGaussian constructs eliminate the traditional decoupling of geometry and simulation, instead using 3D Gaussian kernels as joint physical and visual representation elements. The paradigm encompasses explicit Material Point Method (MPM) integration in visual representations, hybrid simulation-rendering pipelines, data-driven and generative content creation, as well as uncertainty-aware physical reasoning.

1. Foundations: 3D Gaussian Splatting with Physical Integration

PhysGaussian extends standard 3D Gaussian Splatting (3DGS)—where static 3D scenes are rendered as clouds of ellipsoidal Gaussians—by enriching each Gaussian with physically meaningful attributes and evolving these according to continuum mechanics. A canonical formulation represents a scene as a set of primitives

$G_p(X) = \exp\left(-\frac{1}{2}(X - X_p)^\top A_p^{-1}(X - X_p)\right),$

with $X_p$ the undeformed center and $A_p$ , the symmetric positive-definite spatial covariance tensor. Each Gaussian is enhanced with mass ( $m_p = \rho_p V_p^0$ ), velocity $v_p(t)$ , elastic deformation gradient $F_p^E(t)$ , and additional mechanical stress and volumetric information, which evolve under MPM-steered simulation as discrete "material points" (Xie et al., 2023).

Key dynamical variables—position, velocity, deformation, and stress—are propagated in time via a discretized update loop. The opacity and radiance coefficients (e.g., for fast splatting-based rendering) are held fixed or are updated via physically informed transforms (such as spherical-harmonic rotation by local deformation).

A defining characteristic is the strict identity between simulation and rendering representations: the Gaussians used as discrete MPM material points are identically those rendered for view synthesis, embodying 'what you see is what you simulate' (WS $^2$ ) (Xie et al., 2023).

2. Algorithmic Framework: Material Point Method Coupled to Gaussian Kernels

PhysGaussian employs a variant of the Material Point Method for time-stepping dynamics. The key discrete update loop is:

Particle-to-Grid (APIC transfer): Mass and momentum from each Gaussian/material point $p$ $p$ are projected onto grid nodes $i$ $i$ via $C^1$ $C^{1}$ B-spline interpolation (Xie et al., 2023).
- $X_p$ 0
- $X_p$ 1
Grid Update (Explicit Euler): Velocities at each grid node are updated with total internal (material) and external (gravity) forces:
- $X_p$ 2
- Stress updates are governed by the constitutive law, e.g., corotated elasticity
$X_p$ 3
Grid-to-Particle: Velocities, positions, and deformation quantities are interpolated back to Gaussians.
- Velocity: $X_p$ 4
- Position: $X_p$ 5
- Deformation update: $X_p$ 6
- Plastic return-mapping and stress: $X_p$ 7, $X_p$ 8
Covariance and SH Update: The Gaussian's covariance is updated via the local affine transform:
- $X_p$ 9
- Spherical harmonics used for radiance are rotated according to polar decomposition.
Rendering: Each updated ellipsoid is rendered using a standard 3DGS splatting pipeline: ellipsoid projection, alpha-compositing, SH-based shading, and per-pixel occlusion-aware integration.

A rate-form update is optionally employed for smooth covariance evolution: $A_p$ 0

3. Physical Modeling and Constitutive Laws

The PhysGaussian platform supports a wide range of continuum material models, adapted by selecting suitable constitutive energy densities $A_p$ 1 and plastic yield surfaces:

Elastic Solids: Fixed-corotated and Neo-Hookean energy (hyperelastic)
Plastics/Metals: von Mises plasticity for ductile response, Drucker–Prager for granular materials (Xie et al., 2023)
Non-Newtonian Fluids and Pastes: Herschel–Bulkley models, viscoplastic behavior
Granular and Brittle Fracture: Fracture and fragmentation emerge naturally as MPM particles detach under stress

Material definitions are plug-and-play through the specification of energy density and plastic return-mapping; e.g., $A_p$ 2 is chosen per material for modeling realistic plastic flow or yielding.

Internal filling via opacity-controlled volumetric support is essential. Without interior Gaussians to bestow volumetric integrity, simulated objects collapse under self-weight in gravity (Xie et al., 2023).

4. Impact, Applications, and Benchmarking

PhysGaussian achieves real-time or near real-time simulation and rendering of complex materials on commodity hardware. For typical NeRF reconstructions and synthetic scenes, simple dynamics run at 25–36 fps, and more complex (granular, plastic) scenes at 5–10 fps on an Intel i9-10920X + RTX 3090 (Xie et al., 2023).

Quantitative evaluation includes:

Method	Cross-View PSNR (lattice deformation)	Mean FPS (simple/complex)	Visual Fidelity Features
PhysGaussian	+1–5 dB above NeRF-Editing, PAC-NeRF	25–36 / 5–10	Large-strain fidelity, no meshing needed
PhysGaussian (Ablat.)	Drop w/o extensible Gaussians/SH rot	–	Shape collapse/artifacts

Ablation studies confirm necessity of extensible Gaussians and physically-coherent SH lighting (Xie et al., 2023).

Applications extend to generative novel-view synthesis, content creation for AR/VR, robotic simulation, fracture mechanics, and datasets spanning NeRF, consumer, and synthetic BlenderNeRF scenes.

Feed-Forward Synthesis: PhysGM predicts both 3D Gaussians and their physics parameters directly from a single image and synthesizes 4D dynamics via MPM and rendering in under a minute, using probabilistic modeling and direct preference optimization (Lv et al., 19 Aug 2025).
Implicit Integration: i-PhysGaussian couples 3DGS with a fully implicit MPM solver, yielding 20× larger stable time steps and superior structural coherence in challenging dynamical regimes (Cao et al., 19 Feb 2026).
Generalized Constitutive Gaussians: OmniPhysGS models each primitive as a mixture of 12 domain-expert hyperelastic and plastic submodels, enabling heterogeneous, multi-material, and realistic scene dynamics (Lin et al., 31 Jan 2025).
Fracture and Volumetric Interiors: GaussianFluent introduces rapid interior densification and a continuous-damage MPM for brittle fracture, supporting volumetric textures and realistic multistage fragmentation (Huang et al., 14 Jan 2026).
Physical Property Inference: PhysGS adopts Bayesian inference over Gaussians to estimate dense physical property fields (friction, hardness, density) with epistemic/aleatoric uncertainty, supporting robotics and uncertainty-aware planning (Chopra et al., 23 Nov 2025).
PINN-inspired PDE Solvers: Physics-Informed Gaussians combine adaptive Gaussian basis functions with shallow MLPs as mesh representations, achieving high accuracy and efficiency in physics-informed PDE solution (Kang et al., 2024).
MRI Super-Resolution: PhysGaussian for MRI employs explicit Gaussians with tissue-physical parameters and physics-based volume rendering for efficient, high-fidelity, zero-shot MRI SR (Liu et al., 10 Mar 2026).

6. Design Principles and Methodological Implications

PhysGaussian establishes a paradigm where geometry, simulation, and rendering are unified at the primitive level. This is in contrast to classical pipelines that require explicit geometric meshing (triangles, tetrahedra), grid contraction, or remeshing. Notable principles include:

Physical–Visual Identity: The discrete set of Gaussians simultaneously encodes mechanical, kinematic, and radiometric properties; no conversion or loss between physical and visual domains.
Meshless/Adaptive Discretization: The use of unstructured, adaptively filled Gaussians allows handling of large strains, fracture, volumetric flows, and material transitions without topological constraints or artifacts such as mesh tangling.
Composability and Modularity: Constitutive behavior is a local (often learnable) property per Gaussian or per patch, permitting heterogeneity and refinement.
Efficiency: Tight coupling streamlines memory, allows for single-stage GPU-based execution, matches or exceeds performance of grid-based methods in both fidelity and compute time, and sidesteps many traditional bottlenecks.
Differentiability and Learnability: The entire pipeline is often fully differentiable, supporting end-to-end differentiable simulation, learning (e.g., from video or text), and uncertainty quantification (Xie et al., 2023, Lv et al., 19 Aug 2025, Chopra et al., 23 Nov 2025).

7. Limitations, Current Challenges, and Future Directions

PhysGaussian frameworks require physically consistent, volumetrically filled Gaussian sets; hollow (surface-only) representation collapses under load.
Explicit MPM integration (as in the original PhysGaussian) can be sensitive to stability at large time steps; implicit variants (i-PhysGaussian) enhance stability at increased per-step cost (Cao et al., 19 Feb 2026).
Constitutive law specification and parameterization may require manual tuning; automatic parameter identification, spatially-varying materials, and more sophisticated learning remain active areas (Huang et al., 14 Jan 2026, Lv et al., 19 Aug 2025).
Real-time performance at scale (≫10⁸ Gaussians) and under highly heterogeneous or articulated configurations remains challenging.
Integration with richer modalities (polarization cues (Wu et al., 14 Mar 2026), multimodal physical priors (Lin et al., 31 Jan 2025), or uncertainty-aware perception (Chopra et al., 23 Nov 2025)) illustrates the extensible nature and ongoing evolution of the framework.

In sum, PhysGaussian and its derivatives define a powerful, unifying approach to generative physical dynamics and rendering, blending explicit, meshless representations, rigorous continuum mechanics, and high-fidelity computer graphics within a single, differentiable, and scalable computational paradigm (Xie et al., 2023).