Parameter-Robust MPPI for Safe Online Learning of Unknown Parameters

Abstract: Robots deployed in dynamic environments must remain safe even when key physical parameters are uncertain or change over time. We propose Parameter-Robust Model Predictive Path Integral (PRMPPI) control, a framework that integrates online parameter learning with probabilistic safety constraints. PRMPPI maintains a particle-based belief over parameters via Stein Variational Gradient Descent, evaluates safety constraints using Conformal Prediction, and optimizes both a nominal performance-driven and a safety-focused backup trajectory in parallel. This yields a controller that is cautious at first, improves performance as parameters are learned, and ensures safety throughout. Simulation and hardware experiments demonstrate higher success rates, lower tracking error, and more accurate parameter estimates than baselines.

• The paper presents PRMPPI, integrating SVGD-based parameter updates with dual-path MPPI to guarantee safety in the presence of unknown parameters.
• It employs Conformal Prediction to enforce joint chance constraints, offering probabilistic safety guarantees over entire control trajectories.
• Empirical results on quadrotor and cartpole tasks demonstrate rapid parameter convergence, near-perfect safety rates, and improved control performance.

Parameter-Robust MPPI for Safe Online Learning of Unknown Parameters

Introduction and Motivation

Safe reinforcement learning and optimal control in robotics often fail under significant parametric uncertainty, especially when the plant model contains critical but undetermined parameters, such as mass, payload length, or friction. Traditional domain randomization mitigates this by treating a static model distribution, but leads to conservatism or brittleness, failing to exploit the stream of real-world observations. Although Bayesian online parameter estimation methods have emerged, most approaches decouple learning from control, neglect epistemic safety guarantees over the entire prediction horizon, or collapse parameter distribution into point estimates, defeating robust design goals.

The paper "Parameter-Robust MPPI for Safe Online Learning of Unknown Parameters" (2601.02948) addresses these deficiencies by developing a parameter-robust Model Predictive Path Integral (PRMPPI) control framework. This method handles arbitrary, non-Gaussian (and possibly multimodal) parameter beliefs through Stein Variational Gradient Descent (SVGD), propagates parameter uncertainty explicitly during online learning, and enforces joint chance constraints using Conformal Prediction to provide probabilistic safety over full trajectories.

Figure 1: Hardware experiment with the Crazyflie 2.1 quadrotor and an unknown-length suspended payload tracking a square trajectory while avoiding obstacles and refining its parameter belief online.

Methods

SVGD-Based Parameter Belief Updates

The parameter uncertainty is represented nonparametrically as a particle set and updated using SVGD, which deterministically evolves particles to minimize the KL-divergence to the true posterior. This update avoids degeneracy issues inherent to particle filters and supports multimodal, non-Gaussian posteriors, which are intractable for classical Bayesian filters. Each particle update leverages: (1) the collected state transitions (likelihood term via current dynamics), and (2) a kernel density estimation (KDE) prior for smooth, differentiable updates.

Conformal Prediction for Safety Guarantees

To provide tractable, distribution-free probabilistic safety, the method uses Conformal Prediction (CP). For any control trajectory, multiple parameter samples drawn from the belief are propagated through the dynamics, and safety is evaluated using a non-conformity score (the negative distance to constraint violation over the trajectory). Through order statistics and a sample complexity that depends only on the risk parameter $\delta$, CP certifies a lower bound on the probability that the trajectory remains safe across all time steps.

Dual-Trajectory MPPI (Nominal + Robust Optimization)

PRMPPI integrates parameter learning and control by embedding the evolving parameter belief within Model Predictive Path Integral (MPPI) control. Unlike standard MPPI, which only seeks cost minimization, the proposed approach performs parallel optimization of:

• Nominal trajectory: Weighted for performance objectives, penalized for any safety violations (soft-constrained).
• Robust backup trajectory: Optimized solely to maximize the lower bound on safety (as per CP robustness metric).

At runtime, if the nominal trajectory fails the joint chance constraint (insufficient safety probability), control reverts to the robust trajectory as a fallback, ensuring persistent feasibility and safety regardless of parameter learning progression.

Figure 2: In simulation, the quadrotor tracks a circular path (black), while the propagated belief (blue particles + red KDE contours) adapts online. Unsafe regions (red) are actively avoided as belief improves.

Experimental Evaluation

Simulation Benchmarks

Simulations were conducted on safe-control-gym's nonlinear cartpole and quadrotor tasks. Parameter randomizations involved significant uncertainties: up to $10\%$ variation for cartpole masses and $50\%$ for quadrotor masses and inertia. The primary metrics were trajectory RMSE, safety constraint success rate (SR), and parameter estimation accuracy (PA).

Results across 100 random runs show that PRMPPI achieved perfect or near-perfect SR in all cases, outperforming baselines such as static robust MPPI (which suffered from high cost and, in the quadrotor case, safety failures due to over-conservatism), GPMPC (which was limited by slow and data-dependent generalization), and a variant using UKF (which suffered reduced parameter accuracy due to unimodal Gaussian assumptions). Notably, PRMPPI exhibited rapid and accurate parameter identification (PA $\approx$ 99.5%) and minimal RMSE after only two or three laps.

Figure 3: Hardware results for the quadrotor. (a) Repeated $xy$ trajectories in space/time; (b) evolving parameter beliefs (length/damping); (c) convergence of parameter accuracy and trajectory RMSE; (d) minimal distance to obstacles, demonstrating maintained safety margins.

Hardware Validation

The Crazyflie 2.1 quadrotor was equipped with a suspended payload of unknown length and navigated a constrained environment with tight obstacle clearances. The quadrotor’s task was to follow a square reference path—requiring passing both above and below obstacles—with only a prior interval on payload length and arbitrary (unmodeled) damping.

Despite significant model mismatches (unmodeled aerodynamic downwash, simplified cable dynamics), PRMPPI consistently avoided all collisions, increased path optimality as parameters were inferred, and adapted to exploit new feasible path shortcuts (e.g., flying above or below obstacles only after sufficient parameter confidence was attained). Joint online estimation of length and damping was necessary for best performance and tight safety margins; lack of damping estimation led to overestimated payload length and overly-conservative trajectory planning.

Theoretical and Practical Implications

This work demonstrates that robust, receding-horizon control in the presence of substantial parametric uncertainty is feasible without resorting to static worst-case analysis, provided online parameter learning is integrated with explicit trajectory-level probabilistic safety enforcement. The SVGD-based Bayesian update avoids the sample impoverishment found in SIR or unimodal Kalman-based filters, and CP enables rigorous safety certification without strong distributional assumptions on parameter posteriors. The fallback dual-trajectory structure directly addresses the feasibility breakdowns of classical soft-constrained sampling-based MPC under rapidly varying safety envelopes.

These advances have direct implications for safety-critical robotics and autonomy—where model misspecification and abrupt regime changes are the norm—enabling systems to operate efficiently without pre-emptive manual identification or unnecessary conservatism.

Limitations and Future Directions

The approach’s safety guarantee relies on the representational fidelity of the SVGD particle belief relative to the true posterior. As finite-particle approximations may diverge, this leads to mild conservatism or rare failures that could be remedied with robust extensions or error bounds on SVGD updates. Furthermore, coupling between state and parameter observability (e.g., in uninformative system states like cartpole upright equilibrium) can delay parameter convergence, which may be mitigated by active excitation or experimental design.

Future work may extend this framework to dynamics where parameter and state estimation are further entangled, to partially observable Markov decision processes (POMDPs), and to adaptive high-dimensional systems where fast parallelization of SVGD inference mechanisms and real-time robust trajectory optimization are critical.

Conclusion

PRMPPI provides a tractable, scalable, and effective method for safe online learning and control under parametric uncertainty. By integrating SVGD-based parameter updates, CP-based chance-constrained safety, and dual-mode robust trajectory optimization into MPPI, PRMPPI ensures persistent safety while improving performance as parameter information accumulates. Empirical results suggest strong generality across simulation and hardware, supporting its adoption for robust safety-critical robotics and autonomous systems research.