A Compositional Object-Based Approach to Learning Physical Dynamics (1612.00341v2)

Abstract: We present the Neural Physics Engine (NPE), a framework for learning simulators of intuitive physics that naturally generalize across variable object count and different scene configurations. We propose a factorization of a physical scene into composable object-based representations and a neural network architecture whose compositional structure factorizes object dynamics into pairwise interactions. Like a symbolic physics engine, the NPE is endowed with generic notions of objects and their interactions; realized as a neural network, it can be trained via stochastic gradient descent to adapt to specific object properties and dynamics of different worlds. We evaluate the efficacy of our approach on simple rigid body dynamics in two-dimensional worlds. By comparing to less structured architectures, we show that the NPE's compositional representation of the structure in physical interactions improves its ability to predict movement, generalize across variable object count and different scene configurations, and infer latent properties of objects such as mass.

• The paper introduces the Neural Physics Engine (NPE), a framework that factorizes physical scenes into modular object-based representations to simulate dynamics effectively.
• By decomposing object interactions into pairwise components, NPE achieves superior generalization and accurately infers latent properties like mass with around 90% accuracy.
• Its flexible architecture bridges neural adaptability with symbolic structure, paving the way for advanced applications in model-based planning and reinforcement learning.

A Compositional Object-Based Approach to Learning Physical Dynamics

The paper presents the Neural Physics Engine (NPE), a framework tasked with learning simulators of intuitive physics capable of natural generalization across variable object counts and differing scene configurations. This work addresses a fundamental challenge in simulating physical interactions—developing a model both adaptable like neural networks and structured like symbolic physics engines.

Central to the NPE is a novel approach to physical scene representation. The model factorizes scenes into composable, object-based representations and leverages a neural network architecture that further decomposes object dynamics into pairwise interactions. This dual-layer of factorization—both at the object and interaction levels—empowers the NPE to encapsulate the intrinsic structure of physical dynamics while maintaining flexibility through neural learning.

Neural Physics Engine Architecture

The NPE is characterized by several key features:

• Compositional Architectures: By factorizing scenes into object-based representations, the NPE ensures objects and their interactions are modular, enabling seamless adaptation as object count or scene configuration changes.
• Object Interaction Modeling: The dynamics of objects are represented as pairwise interactions, encapsulated by the compositional structure of the network architecture. This reflects the causal structure seen in physical interactions and results in generalization across diverse scenarios without the need for retraining.
• Object as Primitives: The NPE takes advantage of a strong inductive bias where objects are primitives, drawing parallels to human reasoning where objects and their interactions are fundamental to understanding a physical scene.

Performance Evaluation and Results

NPE's efficacy is validated through a series of experiments involving simple rigid body dynamics in two-dimensional worlds. The results show that NPE not only predicts object motion effectively but also demonstrates superior generalization to environments with varying object counts and configurations. Furthermore, the model is capable of inferring latent properties such as mass during these interactions, showcasing its robustness and adaptability.

In quantitative evaluations, the NPE consistently outperformed baseline models, highlighting the advantages of its architectural composition. For example, tests on variable mass scenarios show that NPE achieves approximately 90% accuracy in inferring object mass, evidencing strong predictive power and generalization across previously unseen tasks and configurations.

Implications and Future Developments

The implications of the NPE span both practical applications and theoretical advancements. Practically, this framework enables more efficient and versatile simulation engines that can adjust to new and varied physical inputs without extensive retraining. Theoretically, the NPE presents a step forward in bridging the gap between structured symbolic reasoning and adaptable neural computation, enriching the landscape of computational physics simulation.

Future research directions may include integrating the NPE with perceptual models to automate extraction of physical properties from visual inputs, thus advancing towards a more holistic physics engine. This integration could fuel advancements in model-based planning and reinforcement learning by providing agents with a physics-centric understanding of their environments.

In summary, the NPE showcases a promising approach to learning physical dynamics by harmonizing symbolic and neural components, thus setting a foundation for more comprehensive and adaptable physics simulation frameworks.