GausSim: Foreseeing Reality by Gaussian Simulator for Elastic Objects

Abstract: We introduce GausSim, a novel neural network-based simulator designed to capture the dynamic behaviors of real-world elastic objects represented through Gaussian kernels. We leverage continuum mechanics and treat each kernel as a Center of Mass System (CMS) that represents continuous piece of matter, accounting for realistic deformations without idealized assumptions. To improve computational efficiency and fidelity, we employ a hierarchical structure that further organizes kernels into CMSs with explicit formulations, enabling a coarse-to-fine simulation approach. This structure significantly reduces computational overhead while preserving detailed dynamics. In addition, GausSim incorporates explicit physics constraints, such as mass and momentum conservation, ensuring interpretable results and robust, physically plausible simulations. To validate our approach, we present a new dataset, READY, containing multi-view videos of real-world elastic deformations. Experimental results demonstrate that GausSim achieves superior performance compared to existing physics-driven baselines, offering a practical and accurate solution for simulating complex dynamic behaviors. Code and model will be released. Project page: https://www.mmlab-ntu.com/project/gausim/index.html .

• The paper introduces GauSim, a neural network-based simulator that integrates continuum mechanics to capture realistic elastic object deformations.
• It employs Gaussian Splatting and a hierarchical Center of Mass System to reduce computational load by approximately 90% while enhancing simulation accuracy.
• Experiments on the READY dataset demonstrate superior velocity estimation and long-term deformation prediction compared to traditional physics-based methods.

GauSim: Registering Elastic Objects into Digital World by Gaussian Simulator

The paper introduces GauSim, a neural network-based physics simulator specializing in the dynamic behavior of real-world elastic objects. Unlike conventional kernel-based simulations that perceive kernels as discrete particles, GauSim employs a continuum mechanics approach, treating each Gaussian kernel as a continuous entity to capture realistic deformations. The crux of this methodology lies in combining continuum mechanics with a hierarchical structure known as Center of Mass Systems (CMS), which facilitates a coarse-to-fine simulation paradigm, significantly enhancing computational efficiency and simulation fidelity.

Methodology and Approach

GauSim's foundation on continuum mechanics distinguishes it from previous works by avoiding typical idealized assumptions. The simulator represents objects through Gaussian Splatting, allowing each kernel to function as a continuous matter piece. To optimize computational resources, GauSim organizes these kernels into CMS, which serve as analogs to larger kernels at varying system levels. This hierarchical approach ensures that simulations begin from the top level—coarser representations—and systematically descend, refining detail as they progress to the kernel level. This organization not only reduces computational redundancy but also enriches the fidelity of the simulations by preserving real-world dynamic attributes.

A significant innovation in GauSim is its enforcement of explicit physics constraints, including mass and momentum conservation. By extending continuum mechanics principles such as mass conservation through the preservation of volume, the simulator assures physically plausible outcomes. These properties are rigorously integrated into the hierarchical dynamics through interpretable formulations, enabling GauSim to inherently fulfill principles of mass and momentum conservation in its predictive algorithms.

Dataset and Experimentation

To validate GauSim's capabilities, the researchers introduce the READY dataset, composed of multi-view videos demonstrating complex elastic deformations of real-world objects like moth orchids, pudding, and toy ducks. By comparing GauSim's performance against physics-driven baselines, the system exhibits superior accuracy and realism in dynamic simulations. Unlike generative models relying solely on video supervision, GauSim's physics constraints ensure predictive compliance with real-world physical laws.

Experimental Results and Implications

Quantitative evaluations indicate that GauSim significantly outperforms traditional physics-based simulators, particularly in estimating initial velocities and maintaining long-term prediction accuracy over complex deformations. The hierarchical CMS framework provides substantial computational efficiency, reducing prediction requirements by approximately 90%. Such architectural efficiency posits GauSim as a robust tool for simulating intricate dynamic phenomena in computer graphics and potentially other fields requiring accurate physical model representations.

Future Directions and Impact

The research opens prospects for potentially applying GauSim beyond its initial domain by broadening its applicability to various object types owing to its shape-agnostic, hierarchical structure. Future exploration could involve expanding GauSim's framework to interact with more complex environmental and material systems, extending its reach to fields like virtual reality and interactive animations.

Ultimately, GauSim presents a transformative method for simulating elastic object dynamics by synergizing continuum mechanics with machine learning, backed by a dataset rooted in real-world complexity. This interplay between theoretical constructs and practical implementations provides a pragmatic solution for bridging the divide between virtual simulations and tangible physical phenomena.