StairMaster: Learning to Conquer Risky Hollow Stairs for Agile Quadrupedal Robots

Abstract: Climbing hollow stairs remains a challenging problem for quadruped robots due to the high risk of leg trapping, severe depth sparsity, and high-frequency depth-sensing noise. In this paper, we propose StairMaster, a novel three-stage reinforcement learning framework for stable locomotion on such extreme discontinuous terrains. Our architecture integrates a Cross-Attention mechanism to extract structural features from noisy depth data, alongside a Spatial-aware Recurrent Unit (SRU) that maintains robust spatio-temporal memory to mitigate perception blind spots. To bridge the sim-to-real gap in depth perception, we propose a high-fidelity sim-to-real depth sensor modeling pipeline that faithfully replicates real-world sensor artifacts. Additionally, we employ a 3D waypoint-guided active perception reward for proactive sensing, alongside hollow gap kinematic and stair edge penalties to ensure precise foothold placement. We successfully deployed StairMaster on a Unitree Go2 robot, demonstrating its ability to conquer hollow stairs with an unprecedented incline of up to 55$<sup>\circ$ through zero-shot transfer. To the best of our knowledge, this is the first RL-based policy to achieve such steep hollow stair climbing in real-world environments. Project Website: https://sivan666666.github.io/StairMaster/.

• The paper introduces an innovative three-stage reinforcement learning framework that overcomes sensor noise, occlusion, and sparse footholds on hollow stairs.
• It combines a CNN-based visuospatial encoder with a Spatial-Aware LSTM to integrate temporal memory, enabling precise foot placement and adaptive pitch control.
• Empirical results show up to 97.5% success in simulation on 55° inclines and robust performance in real-world deployments, outperforming conventional baselines.

StairMaster: Learning to Conquer Risky Hollow Stairs for Agile Quadrupedal Robots

Motivation and Problem Statement

Quadrupedal robots face substantial challenges when navigating hollow stairs, which feature large gaps and reflective surfaces. These terrains present three primary hazards: a heightened risk of leg trapping due to discontinuous threads, severe depth sparsity and noise in sensor readings caused by reflective materials, and persistent visual occlusion as terrain passes beneath the robot. Standard proprioceptive or vision-guided RL frameworks are unable to reliably overcome these obstacles due to partial observability, sensor degradation, and the inability to actively adapt robot posture or foot placement for sparse footholds. As a result, there is a clear need for a robust solution that enables agile quadrupedal robots to navigate such risky real-world staircases.

Figure 1: StairMaster enables a quadrupedal robot to climb real-world $55^\circ$ hollow stairs.

Architecture and Methodology

StairMaster introduces a three-stage RL-based approach incorporating advances in visuospatial encoding, sensor modeling, and reward design:

• Stage 1: Privileged Teacher Policy. Training is conducted in simulation using PPO with access to privileged heightmaps and dense proprioceptive features, allowing rapid expert skill acquisition.
• Stage 2: Student Policy Distillation. The student policy learns to map noisy egocentric depth and proprioception to actions via imitation learning, leveraging Cross-Attention mechanisms and a Spatial-Aware LSTM (SRU) for spatio-temporal memory.
• Stage 3: Fine-Tuning. Direct interaction and online fine-tuning via PPO closes the performance gap, adapting to suboptimal behaviors from distillation.

Figure 2: Overview of StairMaster training framework.

The visuospatial encoder combines CNN-extracted depth features with proprioceptive embeddings using Multi-head Cross-Attention. The SRU extends temporal memory with spatial alignment, implicitly tracking terrain topology even through visual occlusion. To bridge the sim-to-real gap, StairMaster incorporates a high-fidelity depth noise modeling pipeline, simulating sensor artifacts including additive noise, hole noise, edge noise, stereo matching errors, and vibration-induced pixel shifts.

Figure 3: Pipeline of high-fidelity sim-to-real depth noise modeling.

Reward Design for Hollow Stairs

StairMaster employs three customized reward mechanisms tailored to hollow stairs:

• 3D Waypoint-Guided Active Perception ($r_{pitch}$): This term penalizes deviation from the desired pitch angle aligned with future stair thread waypoints, enabling proactive trunk pitching and early geometric awareness.
• Hollow Gap Kinematic Penalty ($r_{hollow}$): It applies a penalty when any foot enters hollow regions, enforcing high-clearance swing trajectories.
• Stair Edge Penalty ($r_{edge}$): It discourages footholds too close to stair edges, prioritizing central landing and stable support.

Figure 4: Designed reward for hollow stairs.

These rewards directly address key failure modes: torso-trapping, foot-trapping, and edge-slipping. A terrain curriculum and domain randomization further improve robustness and agility during training.

Experimental Results and Ablation Analysis

Extensive simulation and real-world experiments validate StairMaster's superiority over baselines. In simulation, blind proprioceptive policies (HIMLoco) and vision-based baselines (Extreme Parkour) fail on hollow stairs due to inability to infer sparse footholds or adapt posture for occlusion. StairMaster achieves a $97.5\%$ success rate on $55^\circ$ stairs, outstripping all baselines by wide margins. Ablation studies show that both $r_{pitch}$ and $r_{foothold}$ are critical for collision avoidance and successful traversal. Depth noise robustness is confirmed, with the framework maintaining $99.8\%$ average reached steps under up to $200\%$ sensor noise injection, while ablated models degrade severely.

Figure 5: Classical failure cases in simulation.

Figure 6: Terrain level comparison during training.

Figure 7: Experiment results under different depth noise levels.

Sim-to-Real Transfer and Real-World Deployments

Physical validation was performed using a Unitree Go2 quadrupedal robot equipped with an Intel RealSense D435 depth sensor. On $r_{pitch}$0 hollow stairs, StairMaster achieves $r_{pitch}$1 success; on $r_{pitch}$2, $r_{pitch}$3 success is recorded—a figure unmatched by all baselines. The robot displays active pitch control, accurate foot placement, and fluid motion under severe depth sparsity, evidencing effective visuospatial memory and reward-driven behavioral adaptation.

Figure 8: Snapshots of real-world experiments on the $r_{pitch}$4 hollow stairs.

Figure 9: Snapshots of real-world experiments on the steep $r_{pitch}$5 hollow stairs.

Implications and Future Directions

StairMaster provides a methodologically rigorous and practically validated solution for quadrupedal navigation on hazardous discontinuous terrains. The architectural innovation in multimodal encoder and SRU, combined with high-fidelity sensor artifact simulation and tailored reward shaping, enables unmatched sim-to-real transfer and performance on risky stairs. This approach generalizes to scenarios with severe partial observability and noisy perception, offering new avenues for RL policy design under sparse exteroceptive inputs.

Practically, StairMaster enables safe inspection tasks in industrial environments, expanding the operational envelope of quadrupedal robots in real-world facilities such as power plants and construction sites. Theoretically, it advances RL for partial observability by demonstrating effective spatial memory integration and domain adaptation.

Future directions include enhancing sim-to-real transfer via improved motor modeling, incorporating RGB input for richer environmental awareness, and extending to more complex multi-modal terrains or industrial constraints.

Conclusion

StairMaster establishes a robust RL framework for agile quadrupedal climbing of hollow stairs, combining architectural advances and training innovations to address sensor noise, occlusion, and foothold sparsity. Its sim-to-real effectiveness is empirically validated on extreme inclines. This work points to new RL policy designs for high-risk, discontinuous terrain navigation, and paves the way for further integration of multimodal perception and advanced memory architectures in legged robotics.