Diffusion-Inspired MPC for Legged Robots

• The paper introduces DIAL-MPC which leverages a multi-stage diffusion annealing process to refine control trajectories from global exploration to local exploitation.
• It combines iterative stochastic sampling with a dual-loop annealing strategy and warm-start dynamics to achieve robust, high-agility full-order control in real time.
• Empirical results show significant improvements in tracking error, contact compliance, and agility over traditional MPC and RL approaches in diverse, contact-rich scenarios.

Diffusion-Inspired Annealing for Legged Model Predictive Control (DIAL-MPC) describes a model predictive control architecture that leverages diffusion-inspired annealing techniques to address the high-dimensionality, non-convexity, and multi-modality inherent to legged robot locomotion. DIAL-MPC unifies iterative stochastic sampling, trajectory refinement, and dynamic constraint handling to optimize for robust, high-agility full-order control in real time without reliance on offline policy training. This approach is theoretically motivated by connections between path integral sampling-based MPC, score-based diffusion models, and annealing strategies from non-convex optimization.

1. Theoretical Foundation: Diffusion-Style Annealing in Sampling-Based MPC

DIAL-MPC is conceptually grounded in the linkage between Model Predictive Path Integral (MPPI) control updates and single-step score-based diffusion sampling. In conventional MPPI, control sequences $U$ are iteratively refined by adding Gaussian perturbations and reweighting samples by exponential cost likelihoods: $U^+ = U + \Sigma \, \nabla \log p_1(U),$ where $p_1(U) = (p_0 \ast \varphi)(U)$ is a Gaussian-smoothed target distribution $p_0(U) \propto \exp(-J(U)/\lambda)$.

Diffusion-style annealing generalizes this to a multi-step process, initializing with high-variance noise (for global exploration) and progressively reducing the variance schedule (for local exploitation). At each annealing step, the sampling kernel's covariance is decreased (i.e., the distribution "cools"), mirroring the reverse of the Fokker–Planck diffusion process: $$\det(\Sigma^{(i)}) = \exp\left(-\frac{(N - i)}{\beta N}\, d\right)$$ for annealing index $i = 1,\ldots,N$, with $d$ the control dimension and $\beta$ a temperature parameter. This mechanism enables DIAL-MPC to traverse rugged, high-dimensional, non-convex control landscapes by iteratively refining sampled trajectories from global-to-local modes, thus increasing the success rate of finding feasible, high-reward solutions in tasks such as climbing, jumping, or complex gait generation (Xue et al., 23 Sep 2024).

2. Dual Annealing Algorithm and Warm-Start Dynamics

The DIAL-MPC algorithm employs a dual-loop annealing strategy:

• Trajectory-level (outer): The control sequence across the prediction horizon is annealed with a schedule dependent on the global stage of optimization.
• Action-level (inner): Covariance schedules are heterogenous across time steps—in early steps (where solutions have more impact and are updated more frequently) lower noise is used; for later steps (less frequently updated) higher noise is retained to maintain exploration.

The iterative update at each annealing iteration $i$ and time step $h$ is: $$\Sigma^{(i)}_{t+h} = \exp\left[ -\frac{N-i}{\beta_1 N} - \frac{H-h}{\beta_2 H} \right]\, I,$$ where $H$ is the receding horizon, and $\beta_1, \beta_2$ are tunable parameters. Monte Carlo sampling is performed at each stage, and the empirical score

$$\nabla \log p^{(i)}(U) \approx \Sigma^{-1} \frac{\sum_j e^{-J(U + W_j)/\lambda} W_j}{\sum_j e^{-J(U + W_j)/\lambda}}$$

is used to update the control sequence prior to horizon shifting.

Warm-starting—reusing previous action sequences—further improves convergence and temporal consistency, while the repeated stochastic annealing enables robust behavior under multimodality and dynamic environment changes without explicit trajectory rollouts or reward shaping.

3. Performance Metrics and Empirical Outcomes

DIAL-MPC achieves significant improvements in full-order, torque-level legged robot control:

• Tracking error: 13.4× reduction versus vanilla MPPI in unstructured walking tasks ((Xue et al., 23 Sep 2024), Fig. 2).
• Contact compliance: More than double the sustained contact reward compared to MPPI in sequential jumping.
• Agility: Only DIAL-MPC reliably climbs over obstacles exceeding twice the quadruped's height; RL and non-annealed MPCs typically fail in these cases.
• Generalization: Robust performance maintained under model mismatch (e.g., varying payload masses), with no need for retraining.
• Computational efficiency: Real-time capable; full-order optimization executed at 50 Hz on hardware.

The annealing process enables not only higher reward and lower error but also substantial improvements in solution reliability, especially in contact-rich or discontinuous landscapes that defeat standard path integral or gradient-based MPC.

4. Real-World Hardware and Task Applications

DIAL-MPC has been demonstrated for real-world quadruped robots such as Unitree Go2, with deployment on tasks including:

• Precision jumping with payloads: Leap onto $10\,\text{cm}$-diameter platforms with $7$–$10\,\text{kg}$ payload, using online optimization only, maintaining high tracking accuracy.
• Crate climbing and complex terrain: Solves for full-contact climbing motions over obstacles more than twice the robot’s height, outperforming both RL policies and traditional sampling-MPCs.
• Generalization across tasks: No task-specific retraining required; trajectories and gaits can flexibly adapt as soon as the instantaneous dynamics model changes (e.g., due to attached payload or terrain variation).

Tested systems leverage only online optimization, thus avoiding lengthy RL policy training and associated sim-to-real transfer gaps, making DIAL-MPC suitable for time-critical or rapidly-changing tasks.

5. Relation to Diffusion-Based and Sampling MPC in the Literature

DIAL-MPC’s diffusion-inspired annealing methodology extends the theoretical analysis connecting MPPI and single-step diffusion (Xue et al., 23 Sep 2024), which aligns with contemporary advances in diffusion-based planning such as Diffusion-MPC (Huang et al., 5 Oct 2025), Diffusion-Based Approximate MPC (Julbe et al., 6 Apr 2025), and diffusion-sampling control policies (Huang et al., 30 Apr 2024, Qin et al., 8 Jul 2025).

Distinctive features:

• No offline diffusion policy training: In contrast to DiffuseLoco or DMLoco, DIAL-MPC performs all optimization online, using sampling-based annealing rather than a neural policy.
• Addresses coverage-convergence trade-off: The multistage annealing kernel schedule systematically interpolates between global exploration (essential for escaping poor minima in contact-rich scenarios) and local exploitation (yielding accurate, stable fine motion control).
• Scalability to full-order models: Real-time full-order dynamic control is achieved without reducing to centroidal or simplified models, allowing for fine manipulation of all system degrees of freedom and contacts.

These elements make DIAL-MPC especially suitable in settings where hardware, environmental, or mission constraints change unpredictably and offline RL pretraining is infeasible or insufficient.

6. Comparison to Classical and Modern MPC Methods

Traditional MPC approaches for legged robots often require reduced-order dynamic models to achieve real-time feasibility, are vulnerable to entrapment in local minima, and are limited in their exploration capabilities in highly non-convex spaces. Sampling-based MPCs (e.g., MPPI, path integral control) historically provide better exploration but suffer from large sample variance, poor scaling, and failures in contact or obstacle-rich environments.

DIAL-MPC directly tackles these obstacles via:

• Annealed global-local search: Incremental reduction of sampling noise enables convergence to high-quality local minima after broad initial exploration.
• Empirical superiority in challenging scenarios: Outperforms both training-heavy RL approaches and path integral MPC in dexterous whole-body maneuvers, such as high-jump or sensorimotor climbing without prior examples.

7. Limitations and Future Research Directions

DIAL-MPC’s real-time online optimization remains computationally intensive for very high-dimensional robots or extremely long prediction horizons. Task-specific temperature/covariance scheduling and score estimation hyperparameters require careful tuning to ensure global coverage without excessive variance. Integrations with distributed optimization (Amatucci et al., 18 Mar 2024), online learning of residual dynamics (Zhou et al., 17 Oct 2025), and interactive online reward fine-tuning (Huang et al., 5 Oct 2025) represent fertile avenues for future work. The algorithm’s training-free, annealing-driven design makes it particularly promising for on-the-fly adaptation in field robotics and dynamic environments where classical methods or reinforcement learning controllers are insufficient.