Shield-Loco: Shielding Locomotion Policies with Predictive Safety Filtering

Abstract: Reinforcement learning (RL) policies enable dynamic legged locomotion but lack mechanisms to avoid violations of safety constraints that are absent during training. Large-scale offline safe learning is impractical for covering all edge cases. Existing safety frameworks either rely on reduced-order models that cannot reason about whole-body behaviors or require conservative recovery controllers that degrade task performance. We propose a predictive safety filter that post-hoc filters the nominal contact locations fed to the RL policy. When a collision is predicted, a sampling-based optimizer asynchronously searches for safer contact sequences using a full-physics model, while a learned value function bootstraps long-horizon returns. Our three algorithmic components (geometric projection of sampled contacts, momentum-augmented updates, and replica-exchange) make the optimization tractable in a discontinuous contact landscape. We validate the filter on a quadruped robot in dense, cluttered environments, both in simulation and in the real world, showing substantial reductions in safety violations with minimal deviation from the nominal input.

• The paper introduces a post-hoc predictive safety filter that optimizes desired foot contact locations through full-physics MuJoCo rollouts, allowing a frozen RL policy to avoid obstacles without modifying torque control.
• The method combines geometric SDF projection, momentum-augmented MPPI or CEM optimization, replica exchange, and a learned terminal value function; replica exchange achieved the lowest tracking cost of 0.0086 in simulation.
• Hardware tests on a Unitree Go2 confirmed stable asynchronous operation, but residual collisions from swing-leg motion, stale plans, imperfect obstacle knowledge, and the lack of formal safety guarantees remain important limitations.

Overview

Shield-Loco addresses a persistent gap between the dynamic capability of contact-conditioned reinforcement learning (RL) locomotion policies and their inability to enforce safety constraints that were absent during training. The authors propose a post-hoc predictive safety filter that intervenes not on joint torques or reduced-order state estimates, but on the desired foot contact locations fed to a frozen RL policy. When a rollout of the nominal contact plan predicts a violation, an asynchronous sampling-based optimizer searches for safer contact sequences using full-physics rollouts (MuJoCo MJX), with a learned terminal value function bootstrapping long-horizon returns. The framework is validated on a Unitree Go2 quadruped in dense cluttered environments, both in simulation and on hardware.

The central design claim is that contact locations form a kinematic abstraction expressive enough to steer whole-body collision avoidance while remaining tractable for online sampling-based optimization. Because the RL policy smoothly tracks contact targets at 50 Hz, the safety optimizer can run asynchronously at 3 Hz without destabilizing the low-level controller — a decoupling the authors argue is essential given the cost of full-physics rollouts.

Method

The filter operates in a receding-horizon loop: it first rolls out the nominal contact sequence $\{\bar{p}_{t+\tau}\}$ over horizon $H$ using the closed-loop dynamics $s_{t+1} = f(s_t, \pi(s_t, p_t))$; if no violation is predicted, the nominal contact is passed through unchanged. Otherwise, it solves

$$\max_{p_0,\dots,p_{H-1}} \mathbb{E}\Big[\sum_{t=0}^{H-1}\gamma^t r(s_t,a_t) + \gamma^H V(s_H)\Big]$$

subject to safety constraints $g(s_t, p_t) \leq 0$, warm-started from the nominal plan. Safety is enforced via two soft penalties evaluated on actual simulated foot positions rather than commanded targets: a signed-distance-function (SDF) margin penalty against obstacle primitives (spheres, cylinders, boxes, capsules), and a large penalty $M\,\mathbf{1}_{\mathrm{col}}$ triggered by any whole-body link contact reported by MuJoCo. A notable assumption is that obstacle geometry and poses are known a priori and perfectly known in simulation; online perception under uncertainty is explicitly deferred.

Three algorithmic components make optimization tractable in the discontinuous contact landscape:

• Geometric projection: sampled contacts are projected onto the collision-free set $\mathcal{F}$ by solving a small QP built from linearized SDF constraints. Projection is applied both during rollouts ("in") and to the final optimized plan before execution ("out").
• Momentum-augmented updates: derived from accelerated proximal natural gradient descent via the Bayesian learning rule, yielding Nesterov-style extrapolation of both mean and covariance updates, applicable verbatim to MPPI soft weights or CEM elite weights.
• Replica exchange: parallel tempering over $L=20$ MPPI replicas with staggered inverse temperatures, swapping adjacent replicas via a Metropolis–Hastings criterion to escape shallow local minima.

The terminal value function $V$ is a small MLP trained offline via n-step TD learning on trajectories collected by the optimizer itself, aligning the bootstrap target with the planner's cost landscape.

Simulation results

Against CBF and HJ reachability baselines built on unicycle approximations, all three optimizer variants achieve the best safety-tracking trade-off, reducing actual violations to roughly 30 per episode while maintaining low tracking costs. Replica Exchange attains the lowest tracking cost ($0.0086$). Both model-based baselines reduce planner violations relative to the nominal plan but incur higher tracking costs and more actual collisions, supporting the paper's claim that reduced-order approximations are inadequate for dense cluttered scenes.

Several results deserve emphasis because they complicate a naive reading:

• Planned vs. actual violation gap: actual violations remain substantially higher than planner violations across all methods. The authors attribute most residual collisions to swing-leg trajectories and intermediate steps taken by the RL policy despite safe contact targets — a failure mode that contact-space optimization cannot directly address.
• Compute budget trade-off: increasing iterations $H$0 or horizon $H$1 often increases actual collisions, as plans become stale in the asynchronous setting. The authors identify this as a fundamental trade-off of asynchronous predictive safety filters; moderate budgets ($H$2, $H$3) perform best.
• Momentum ablation: non-zero momentum yields consistent tracking-cost gains, with $H$4 best overall; MPPI benefits most at low iteration counts (~40% improvement at $H$5), while high momentum ($H$6) increases variance.
• Projection ablation: projection is the most impactful component; applying it only during rollouts does not guarantee feasibility of the final plan, since iterative reweighting can collapse the sampling distribution toward an infeasible mode. The final "out" projection is critical.
• Value function ablation: enabling $H$7 cuts tracking cost substantially (e.g., MPPI: $H$8) but yields only marginal gains in violations, again pointing to the swing-phase limitation.
• Sample budget: increasing $H$9 from 32 to 512 improves tracking and planner violations with diminishing returns beyond $s_{t+1} = f(s_t, \pi(s_t, p_t))$0, but actual violations do not decrease monotonically — only Replica Exchange continues improving at $s_{t+1} = f(s_t, \pi(s_t, p_t))$1.

Hardware validation

On a Unitree Go2, proprioceptive state is streamed to an external PC running the optimizer over a tethered connection. Keyframe sequences show the filter redirecting a footstep into a gap between objects and steering the body laterally around boxes and poles. Online statistics compare $s_{t+1} = f(s_t, \pi(s_t, p_t))$2 (~200 ms/cycle) versus $s_{t+1} = f(s_t, \pi(s_t, p_t))$3 (~400 ms/cycle): the longer horizon intervenes less aggressively with lower tracking cost, at the price of staleness. All hardware runs completed successfully, empirically validating the asynchronous decoupling — the 50 Hz policy remains stable regardless of optimizer latency.

Limitations and open questions

The paper is candid about several limitations. First, the method provides no theoretical guarantee of optimality or safety: the filtered plan can deviate from the true safe set when the sampler converges to a local optimum, and the geometric projection is a heuristic whose effect on convergence properties is not analyzed formally. Second, residual actual violations stem from the frozen policy's swing-phase behavior, which motivates — but does not yet include — safe online fine-tuning using data collected by the optimizer. Third, although the framework is stated generally for contact-conditioned legged robots, validation is limited to a quadruped on flat terrain; humanoid loco-manipulation, multi-terrain locomotion, and maneuvers requiring active torso twisting fall outside what contact-location optimization alone can capture. Finally, deployment assumes perfectly known obstacle geometry, so integration with online perception under uncertainty remains open.

Conclusion

Shield-Loco demonstrates that intervening on contact locations — rather than torques or reduced-order states — is a viable abstraction for predictive safety filtering of RL locomotion policies, enabling whole-body collision reasoning through full-physics rollouts at practical computational cost. The empirical evidence supports substantial violation reduction with minimal deviation from nominal behavior, but also exposes structural limits: the planned-versus-actual violation gap rooted in policy swing-phase dynamics, the staleness trade-off inherent to asynchronous filtering, and the absence of formal safety guarantees. These gaps define concrete open problems for extending the approach beyond flat-terrain quadrupedal locomotion.