Predictive Sampling: Real-time Behaviour Synthesis with MuJoCo

Abstract: We introduce MuJoCo MPC (MJPC), an open-source, interactive application and software framework for real-time predictive control, based on MuJoCo physics. MJPC allows the user to easily author and solve complex robotics tasks, and currently supports three shooting-based planners: derivative-based iLQG and Gradient Descent, and a simple derivative-free method we call Predictive Sampling. Predictive Sampling was designed as an elementary baseline, mostly for its pedagogical value, but turned out to be surprisingly competitive with the more established algorithms. This work does not present algorithmic advances, and instead, prioritises performant algorithms, simple code, and accessibility of model-based methods via intuitive and interactive software. MJPC is available at: github.com/deepmind/mujoco_mpc, a video summary can be viewed at: dpmd.ai/mjpc.

• The paper introduces MJPC, an interactive framework applying Predictive Sampling in MuJoCo for real-time robotic control.
• It demonstrates that a straightforward zero-order optimization can compete with advanced planners using rapid shooting-based methods.
• The framework accelerates simulation time and enables dynamic task parameter adjustments, facilitating agile robotics research.

Predictive Sampling: Real-time Behaviour Synthesis with MuJoCo

Introduction

"Predictive Sampling: Real-time Behaviour Synthesis with MuJoCo" (2212.00541) introduces MuJoCo MPC (MJPC), an open-source, interactive application designed for real-time predictive control in robotic systems. This framework leverages the physics of MuJoCo to facilitate the development of complex robotics tasks using predictive control algorithms, prominently featuring three shooting-based planners: iLQG, Gradient Descent, and Predictive Sampling. Predictive Sampling stands out as a straightforward, zero-order optimization algorithm that unexpectedly competes well with more complex planners, despite its initial design for pedagogical purposes.

Framework and Tools

MJPC is structured to enhance accessibility to model-based methods, providing an intuitive graphical user interface (GUI) that allows researchers to author tasks interactively. The software is distinct in its ability to accelerate simulation time relative to real-world conditions, enabling patterns of behavior synthesis even on older hardware. The interactive nature of MJPC allows users to modify task parameters dynamically, speeding up the iterative design process substantially.

Figure 1: Interactive GUI showcasing task modules, planner settings, and real-time simulation predictions.

Predictive Sampling

Despite its simplicity, Predictive Sampling offers computational competitiveness in real-time contexts. This planner operates by repeatedly sampling potential action trajectories and selecting paths based on immediate returns relative to the objective function. It utilizes Gaussian sampling for trajectory exploration, maintaining simplicity while displaying robustness in real-time optimization against more sophisticated derivative-based methods.

Numerical Results

The framework's efficacy is demonstrated through simulations of high-dimensional systems performing intricate tasks such as humanoid standing and hand manipulation:

Figure 2: Humanoid standing up off the floor, exemplifying the synthesis of complex behaviors.

In these examples, MJPC's planners were assessed over planning timeframes between 1 and 20 milliseconds, emphasizing MJPC's suitability for agile, dynamic environments requiring rapid adjustments. These tests demonstrate MJPC's adaptability to various distillates and mismatched simulation models without compromising performance.

Applications and Future Work

MJPC presents several practical uses, such as task design, generating datasets for learning-based approaches, and serving as a playground for predictive control researchers. The framework's open-source nature invites extensive community collaboration to advance research in robotics and control theory. Future development may address the inherent myopic limitation of predictive control by integrating learned policies, enhancing value function utilization, and developing hierarchical control systems.

Conclusion

MuJoCo MPC (MJPC) sets a precedent in democratizing access to predictive control methodologies in robotics through a blend of intuitive design and computational efficiency. By leveraging real-time synthesis capabilities, MJPC notably lowers barriers to entry in robotics research, while providing a substantial toolset for the development and evaluation of complex control solutions. The open-source availability of MJPC encourages collaborative refinement and expansion, paving the way for breakthroughs in both theoretical and applied robotics.