A Proprioceptive-Only Benchmark for Quadruped State Estimation: ATE, RPE, and Runtime Trade-offs Between Filters and Smoothers

Abstract: We compare three state-of-the-art proprioceptive state estimators for quadruped robots: MUSE [1], the Invariant Extended Kalman Filter (IEKF) [2], and the Invariant Smoother (IS) [3], on the CYN-1 sequence of the GrandTour Dataset [4]. Our goal is to give practitioners clear guidance on accuracy and computation time: we report long-term accuracy (Absolute Trajectory Error, ATE), short-term accuracy (translational and rotational Relative Pose Error, RPE), and per-update computation time on a fixed hardware/software stack. On this dataset, RPEs are broadly similar across methods, while IEKF and IS achieve a lower ATE than MUSE. Runtime results highlight the accuracy-latency trade-offs across the three approaches. In the discussion, we outline the evaluation choices used to ensure a fair comparison and analyze factors that influence short-horizon metrics. Overall, this study provides a concise snapshot of accuracy and cost, helping readers choose an estimator that fits their application constraints, with all evaluation code and documentation released open-source at https://github.com/iit-DLSLab/state_estimation_benchmark for full reproducibility.

• The paper demonstrates that IEKF and IS achieve lower ATE and RPE compared to MUSE while carefully balancing computation and latency.
• It outlines a rigorous methodology using leg kinematics, IMU, and contact sensors from the GrandTour dataset to assess estimator performance.
• The study highlights practical trade-offs between global trajectory accuracy and short-horizon consistency essential for embedded quadruped control.

Proprioceptive-Only State Estimation for Quadrupeds: Benchmarking Filtering and Smoothing Paradigms

Introduction

This paper, "A Proprioceptive-Only Benchmark for Quadruped State Estimation: ATE, RPE, and Runtime Trade-offs Between Filters and Smoothers" (2605.11674), provides a systematic benchmarking of three recent state estimators for legged robots—MUSE, the Invariant Extended Kalman Filter (IEKF), and the Invariant Smoother (IS)—in a proprioceptive-only regime. The comparison leverages data from the GrandTour Dataset, evaluating estimators in terms of Absolute Trajectory Error (ATE), Relative Pose Error (RPE), velocity RMSE, and computational latency. The work addresses the needs of practitioners who must balance global accuracy, local consistency, and compute cost, particularly for embedded quadruped deployments where exteroceptive sensing is often unreliable or unavailable.

Estimator Algorithmic Structure

All three estimators considered estimate 6-DoF base pose and body velocity from joint encoders, IMU, and foot contact sensors. However, they represent distinct inference paradigms:

1. MUSE (proprioceptive mode): MUSE is a modular, cascaded estimator with a globally stable nonlinear attitude observer (NLO) and eXogenous Kalman Filter (XKF) for orientation and gyro bias, leg-odometry leveraging kinematics and contact, and a filter that fuses IMU integration and leg kinematics [nistico2025muse].

2. Invariant Extended Kalman Filter (IEKF): The IEKF exploits symmetry via right-invariant error representations on Lie groups, leading to state-independent linearized error dynamics. Its measurement update fuses foot kinematics and assumes velocity zero at contacts [hartley2020contact].

3. Invariant Smoother (IS): The IS formulates the state estimation as MAP inference over short fixed-lag windows, using factor-graph optimization with group-affine residuals. It inherits the linearization-invariance property, but as a smoother, trades more computational cost and latency for improved consistency and noise handling [yoon2024invariant].

These estimators are evaluated with all exteroceptive sources disabled, isolating performance in challenging perception-degraded conditions.

Experimental Setup

The evaluation uses the CYN-1 sequence from the GrandTour dataset [frey2026grandtour], comprising 300 meters of outdoor quadruped motion with challenging terrain and sensor conditions. Each estimator is run on a standardized hardware/software stack, and outputs are post-processed with the "evo" package [grupp2017evo], ensuring metric consistency and alignment to KITTI evaluation protocols.

Performance is assessed on three principal axes: global accuracy (ATE), local consistency (RPE), and computational latency, with additional attention to velocity profiles and robustness to poor initialization.

Trajectory and Velocity Estimation Results

The global and path-aligned accuracy of the three estimators is visualized by directly comparing trajectories against ground truth.

Figure 1: GT vs. estimated trajectories for MUSE, IEKF, and IS on the CYN-1 dataset.

The overlays demonstrate broadly consistent estimates, with modest endpoint drift primarily evident in MUSE.

For velocity estimation, all estimators track true base velocity profiles closely across the test sequence.

Figure 2: GT vs. estimated velocities for all methods, showing accurate linear and angular velocity estimation.

A quantitative summary follows:

Metric	MUSE	IEKF	IS
ATE [m]	2.27	1.41	1.36
ATE_vel [m/s]	0.88	0.87	0.87
RPE (Δ=1m) [m]	0.072	0.043	0.043
RPE (Δ=1f) [m]	0.00067	0.00054	0.00197
RPE (Δ=1m) [deg]	0.58	0.57	0.57
RPE (Δ=1f) [deg]	0.0026	0.0026	0.0026

Key observations:

• IEKF and IS attain lower ATE than MUSE, but all methods remain within ≈0.3% trajectory length in path-end deviation.
• Translational RPE over 1m windows is ~40% better for IEKF/IS versus MUSE, indicating better suppression of meter-scale drift in the invariant models.
• RPE at 1 frame favors filters due to faster update rates, with IS less competitive under frame-level noise.

Trajectory and velocity error evolution is further plotted below.

Figure 2: Detailed comparison of velocity and trajectory errors for each approach, illustrating bounded error profiles and close tracking of ground truth.

Computational Efficiency and Latency

A critical practical consideration for embedded quadruped platforms is estimator runtime per iteration (including all sensing, propagation, fusion, and update costs). The following plot summarizes the timing across window sizes:

Figure 3: Computation time per iteration (log scale), showing sub-millisecond latency for MUSE and IEKF, and IS’s window-size-dependent trade-off.

Findings:

• MUSE is fastest (~0.012 ms/iteration), followed by IEKF (~0.02 ms).
• IS latency depends on window size: ~0.11 ms for WS=1, rising to ~0.58 ms for WS=5.
• Smoothers’ lag grows with window length, potentially limiting real-time use in high-rate feedback control.

Robustness to Initialization

The estimators were challenged with initialization containing a large (180°) orientation error, to emulate real-world startup conditions or sensor faults. The orientation estimation convergence over time is plotted below:

Figure 4: Orientation error convergence after large initial misalignment, demonstrating fast recovery for all estimators—MUSE converges faster.

MUSE’s globally stable attitude observer ensures rapid convergence. Both IEKF and IS eventually recover, but with slower error reduction due to their reliance on local linearizations. All approaches preserve estimator stability under these challenging starts.

Practical and Theoretical Implications

The results deliver nuanced guidance for estimator selection:

• Global Path Accuracy: IS and IEKF are preferable where drift minimization is paramount, e.g., for mapping or long-horizon navigation tasks.
• Short-Horizon Consistency: For high-rate control and foot planning, filters like MUSE and IEKF offer superior frame-to-frame consistency and minimal latency, crucial for responsive feedback.
• Computation and Latency: Embedded implementations constrained by cycle time or energy budgets may favor MUSE or IEKF. IS’s smoothing benefits only justify its increased latency on platforms with adequate compute capacity and where delays are tolerable.
• Robustness: When initial state uncertainty is high, globally stable observers such as MUSE’s NLO/XKF provide resilience, though all tested estimators eventually converge.

Contradictory claim: Despite IS’s more sophisticated optimization and regularization, it does not outperform IEKF by substantial margins; in fact, for some short-rate RPE metrics, IEKF or MUSE are superior.

Future Directions

The benchmark establishes a reproducible foundation (pipeline released open-source) for subsequent community studies. Key future areas include:

• Extension to broader datasets incorporating more legged platforms and diverse terrain.
• Integrated benchmarks for closed-loop control and disturbance rejection, to unify perception and actuation assessments.
• Comprehensive analysis under sensor degradations, slip events, and contact failures.
• Optimizing estimators for hardware-level deployability, including quantifying energy efficiency, communication load, and end-to-end system latency.

Conclusion

This study rigorously compares representative filtering and smoothing approaches for quadrupedal proprioceptive state estimation in realistic, open-world scenarios. While all methods deliver robust and accurate performance, trade-offs between local consistency, global path accuracy, latency, and computational expense dictate optimal selection for specific mission requirements. The open-source benchmarking enables reproducibility and standardized evaluation, facilitating methodological advances and practical deployment on legged robots.