Switch: Learning Agile Skills Switching for Humanoid Robots

Abstract: Recent advancements in whole-body control through deep reinforcement learning have enabled humanoid robots to achieve remarkable progress in real-world chal lenging locomotion skills. However, existing approaches often struggle with flexible transitions between distinct skills, cre ating safety concerns and practical limitations. To address this challenge, we introduce a hierarchical multi-skill system, Switch, enabling seamless skill transitions at any moment. Our approach comprises three key components: (1) a Skill Graph (SG) that establishes potential cross-skill transitions based on kinematic similarity within multi-skill motion data, (2) a whole-body tracking policy trained on this skill graph through deep reinforcement learning, and (3) an online skill scheduler to drive the tracking policy for robust skill execution and smooth transitions. For skill switching or significant tracking deviations, the scheduler performs online graph search to find the optimal feasible path, which ensures efficient, stable, and real-time execution of diverse locomotion skills. Comprehensive experiments demonstrate that Switch empowers humanoid to execute agile skill transitions with high success rates while maintaining strong motion imitation performance.

• The paper demonstrates that integrating an augmented skill graph with buffer nodes enables 100% skill switching success rates under diverse locomotion scenarios for humanoid robots.
• The unified whole-body tracking policy, trained with PPO and enhanced with a Foot-Ground Contact Reward, achieves minimal motion errors during complex transitions.
• The online skill scheduler facilitates real-time replanning to recover from disturbances, ensuring seamless transitions and continuous execution of intricate motor tasks.

Switch: Hierarchical Learning of Agile Skill Switching for Humanoid Robots

Problem Formulation and Motivation

Recent progress in RL-driven whole-body control has enabled humanoid robots to execute complex locomotion skills, but practical deployment is impeded by the inability to reliably transition between distinct behaviors. Existing controllers, largely based on goal-conditioned imitation or trajectory tracking, exhibit poor switching performance, often resulting in failures or unnatural movements due to the lack of feasible transition states and vulnerability to disturbances. The “Switch” framework addresses these limitations by formulating skill switching as a state-conditioned reference selection problem, utilizing an augmented skill graph to ensure dynamically feasible transitions and incorporating online replanning for recovery from deviations or perturbations.

Methodological Framework

Switch is constructed as a hierarchical system comprising three principal modules:

1. Skill Graph Data Augmentation: Motion capture data representing multiple skills are retargeted onto the robot. Each frame serves as a graph node, with intra-skill sequence transitions as edges. Cross-skill transitions are established by searching for nearest neighbor states across skills based on pose similarity. Buffer nodes are inserted for transitions with large gaps, facilitating physiologically plausible intermediate states. This augmented graph synthesizes feasible transitions not covered by raw data, mitigating combinatorial trajectory requirements and improving coverage.

   Figure 1: Retargeting and skill graph construction, with cross-skill transitions augmented via similarity and buffer nodes to enhance switching feasibility.

2. Unified Whole-Body Tracking Policy: A single policy is trained via PPO to execute both skills and transitions using the constructed skill graph. Buffer-aware imitation rewards are introduced: during buffer node traversal, supervision is directed toward the target state at the end of the buffer segment, guiding exploration and convergence. Standard imitation rewards are extended with a Foot-Ground Contact Reward (FGR) to explicitly supervise high-frequency contact events critical for agile motion fidelity, especially in complex skills like dancing.
3. Online Skill Scheduler: At runtime, a scheduler monitors the state and tracking errors. On intended switches or substantial deviations (e.g., due to external perturbations), the scheduler performs online graph search (either global, via shortest-path computation, or local via nearest neighbor) to select feasible intermediate references for the policy, enabling robust, real-time recovery and seamless skill transitions.

   Figure 2: The online scheduler automatically replans paths following disturbance-triggered deviation, selecting a get-up segment for recovery and resumption of target execution.

Experimental Evaluation

The Switch system is evaluated on the 29-DoF Unitree G1 humanoid, running an onboard PPO policy. Comparative baselines include a single-skill RL tracking policy, skill graph tracking without buffers, and the state-of-the-art general motion tracker GMT.

Metrics include Skill Switching Success Rate (SSR), Normalized Reward (NR), and multiple error metrics for motion fidelity (global and root-relative body position, joint position/velocity, velocity, acceleration). Tests are conducted across progressive skill switching difficulty (one, two, three consecutive switches) in both perturbed and unperturbed settings.

Figure 3: Quantitative evaluation under perturbed and non-perturbed conditions showing lower full-body tracking errors for Switch, especially in complex transitions and with FGR.

Figure 4: Skill switching success rate stability under random 25 N simulation pushes, evidencing robustness against perturbations.

Numerical Results and Notable Claims

• Perfect Skill Switching Rates: Switch with full augmentation (Skill Graph + Buffers + FGR) achieves 100% SSR under all difficulty levels, while baseline RL policies and GMT collapse to 2–30% as difficulty increases.
• Minimal Motion Errors: Switch consistently retains lower global ($E_{\text{g-mpbpe}}$) and root-relative body position errors (0.075–0.098m) compared to GMT (0.396–0.588m) and buffer-free variants.
• Enhanced Lower-Body Tracking: FGR significantly reduces lower body tracking errors and improves foot-ground interaction, as visually evidenced in coordinated dancing motions.
• Robust Recovery: Online scheduling enables immediate replanning and stable recovery from severe disturbances, maintaining execution continuity where baseline methods fail.

  Figure 5: Visual demonstration of coordinated lower-body movement and improved foot-ground interaction under dancing, surpassing ASAP and GMT baselines.

Theoretical and Practical Implications

Switch’s approach to flexible skill switching via skill graph augmentation and buffer-aware imitation is theoretically significant for scalable multi-skill RL. By synthesizing physiologically plausible transitions and providing real-time state-conditioned planning, the system bypasses the need for exhaustive data collection and bridges the kinematics-dynamics gap, which has historically impeded robotic skill composition.

Practically, the methodology provides a template for deploying humanoid robots in dynamic, real-world environments where rapid adaptive switching and disturbance recovery are critical. The incorporation of explicit contact rewards and buffer nodes enhances agility, enabling higher fidelity execution of skills requiring complex whole-body coordination.

Future Directions

The architecture suggests several avenues for future research:

• Graph-based skill composition can be extended to thousands of behaviors, further scaling flexible motor control.
• Learning or synthesizing buffer node transitions for unrepresented skills could refine transition dynamics.
• Integration of vision, language-conditioned command interfaces in the scheduler layer could enable more responsive and generalizable deployment.
• Multi-agent collaboration and environment-adaptive planning are feasible extensions to the system.

Conclusion

Switch achieves seamless and robust skill switching in humanoid robots using a hierarchical framework combining skill graph data augmentation, buffer-aware imitation learning, and online search-based scheduling. Through systematic construction and evaluation, the method demonstrates perfect switching rates, low tracking errors, and resilience to disturbances across a range of locomotor tasks. It constitutes a foundational advance for the practical deployment of agile, multi-skill humanoid robots in uncontrolled, dynamic settings.