Graph networks as learnable physics engines for inference and control

Abstract: Understanding and interacting with everyday physical scenes requires rich knowledge about the structure of the world, represented either implicitly in a value or policy function, or explicitly in a transition model. Here we introduce a new class of learnable models--based on graph networks--which implement an inductive bias for object- and relation-centric representations of complex, dynamical systems. Our results show that as a forward model, our approach supports accurate predictions from real and simulated data, and surprisingly strong and efficient generalization, across eight distinct physical systems which we varied parametrically and structurally. We also found that our inference model can perform system identification. Our models are also differentiable, and support online planning via gradient-based trajectory optimization, as well as offline policy optimization. Our framework offers new opportunities for harnessing and exploiting rich knowledge about the world, and takes a key step toward building machines with more human-like representations of the world.

• The paper introduces a novel framework that models physical systems as graphs for accurate prediction and control.
• The methodology employs a forward model for one-step prediction and recursive trajectory rollout, demonstrating strong zero-shot generalization across diverse systems.
• Experimental results show lower rollout errors and improved sample efficiency using MPC and reinforcement learning compared to baseline models.

Object- and Relation-Centric Representations for Understanding Physical Systems

The paper presents a novel framework utilizing Graph Networks (GNs) for modeling and controlling complex dynamical systems. Unlike traditional approaches that embed physics knowledge a priori, this approach learns object-centric and relation-centric representations, leading to efficient and accurate predictions in both simulated and real-world settings.

Overview of the Approach

The core contribution is the formulation of physical systems as graphs, where bodies and joints are represented as nodes and edges, respectively. This representation allows the extension of Graph Neural Networks (GNNs) to capture the dynamics of a system effectively. The introduced GN framework provides a mechanism for inductive bias that leverages these graph representations to predict and control system behaviors across various dynamical settings.

Forward and Inference Models

The forward model utilizes GNs for one-step predictions and supports trajectory rollouts through recursive application. Emphasizing robust generalization, the model was trained across eight distinct physical systems, showing high accuracy in predictive capabilities compared to constant and MLP baseline models. The inference model, on the other hand, effectively performs implicit system identification, which is crucial for deploying this framework in partially observable environments.

Experimental Evaluation

Prediction: The forward model demonstrated significant potential in predicting the dynamics of several systems like Swimmers and Cartpole, even when varying the system parameters or structures. Zero-shot generalization, particularly for systems not seen during training, underscores the model's ability to understand underlying compositional rules governing physical interactions.

Control: By employing the forward model in a Model Predictive Control (MPC) schema, the work illustrates its versatility in planning tasks. The MPC, leveraging the GN predictions, performed comparably with baseline approaches. The SVG(1) algorithm was applied for reinforcement learning, improving sample efficiency and outperforming model-free baselines in early performance measures.

Numerical Results

One of the compelling elements of the research is its quantitative analysis. Predictions for systems like SwimmerN showed rollout errors significantly lower than the baselines, even when evaluated on challenging action trajectories. The paper highlights the efficiency with which learned models support complex planning tasks traditionally reserved for manually designed models.

Implications and Future Directions

The implications of this work for both theoretical and practical domains are multifaceted. The approach opens new avenues in leveraging learned representations for model-based planning and decision-making processes, providing an alternative to the rigidity of classical physics engines. Future endeavors could explore scaling these frameworks to more diverse and stochastic environments, including real-world applications that require adapting to unknown dynamics.

Further exploration might involve extending the GN-based approach to facilitate sim-to-real transfer, potentially transforming the deployment of robotics. Additionally, enhancing the model's capacity to handle noise and uncertainty could improve its application in tasks with incomplete data.

In summary, the research delivers an advanced computational paradigm leveraging GNs, positioning itself as a significant step towards machines that understand and manipulate physical environments with a human-like approach to representation and inference.