RoboNaldo: Accurate, Stable and Powerful Humanoid Soccer Shooting via Motion-Guided Curriculum Reinforcement Learning

Abstract: Elite humanoid soccer shooting requires whole-body stability, high-impulse whole-body interactions, and accuracy to targets. Motion tracking-driven reinforcement learning (RL) provides stability in whole-body movement coordination, but a fixed reference makes it hard to adapt to varied ball positions and strike timings; in contrast, task reward-driven RL struggles to explore and discover valid kicks from scratch. We therefore introduce RoboNaldo, a three-stage motion-guided curriculum RL framework for high-impulse humanoid interaction. A single human-kick reference is used as a scaffold and progressively shifts optimization towards shooting performance. The curriculum first learns a stable whole-body kicking prior, then adapts the kick to free-kick settings where the ball is stationary at random positions, and finally extends it to moving-ball shooting through a locomotion-command and kick-trigger interface. A high-level heuristic planner controls this interface during training, while alternative high-level controllers can drive the same low-level policy at inference. In simulation, RoboNaldo demonstrates free-kick shot error 48.6% lower and shoot velocity 2.96x than prior work baselines. In real world on a Unitree G1 with onboard perception, RoboNaldo attains 0.73 m and 0.86 m average target shooting error from 3 m away in free-kick and moving-ball cases, accordingly. And the post-contact ball velocity reaches 13.10 m/s, which is 59-71% of reported professional open-play shot speed. Project page: https://opendrivelab.com/RoboNaldo.

• The paper introduces a staged curriculum reinforcement learning framework that leverages human motion data to build stable, coordinated, and high-impulse soccer shooting skills in humanoid robots.
• The method integrates three phases—motion tracking, task adaptation, and generalization—to progressively enhance shot accuracy, speed, and robustness under dynamic conditions.
• Ablation studies validate the importance of each curriculum stage and specialized reward signals, achieving significant improvements with up to 2.96x higher shot velocity and 80%+ success in real-world trials.

RoboNaldo: Motion-Guided Curriculum Reinforcement Learning for Humanoid Soccer Shooting

Overview

The paper "RoboNaldo: Accurate, Stable and Powerful Humanoid Soccer Shooting via Motion-Guided Curriculum Reinforcement Learning" (2606.11092) introduces a curriculum reinforcement learning framework that produces a unified, robust, and high-performance low-level policy for humanoid soccer shooting. Through a staged learning process guided by human motion data and task-specific rewards, the system achieves accurate, high-impulse soccer striking with real-time perceptual feedback on a full-sized humanoid robot, validated both in simulation and with extensive outdoor hardware trials.

Motivation and Problem Setting

Soccer shooting by humanoid robots encapsulates the intersection of high-velocity end-effector contact, whole-body balance, and temporal precision—an open challenge in robot control. Existing RL-based baselines tend to achieve only subsets of these requirements: motion-tracking approaches provide stable kinematic coordination but cannot generalize strike timing or adapt to new scenarios, while reward-driven RL struggles with credit assignment due to sparse and delayed rewards, often failing to produce coordinated, stable kicks.

RoboNaldo proposes a solution via staged curriculum learning: (1) initialization with a human motion prior for basic balance and coordination, (2) adaptation for accurate, powerful free-kick shooting, and (3) extension to moving-ball soccer through a hybrid interface supporting complex approach and timing decisions.

Curriculum Architecture

RoboNaldo's design is centered on a three-stage curriculum that incrementally incorporates motion prior, task adaptation, and decision-temporal complexity:

Stage 1 (Motion Tracking): The policy is initialized using retargeted human motion capture data, focusing on stable, coordinated kicking without task-specific rewards. This establishes foundational balance and the correct gross body trajectory.

Stage 2 (Task Adaptation): The environment introduces variable ball positions and explicit task rewards for accurate free-kicks. The low-level policy adapts to adjust the interaction point and impulse direction, enabling robust shooting accuracy.

Stage 3 (Generalization): For moving-ball scenarios, the observation space switches to a high-level command interface (locomotion direction, kick-trigger signal), allowing the policy to handle approach, contact timing, and stabilization for dynamic passes. A simple but carefully engineered heuristic high-level planner is used to orchestrate the transition between locomotion and kick phases.

Figure 1: The staged motion-guided curriculum, progressing from motion imitation to adaptive free-kick and moving-ball shooting through increasingly sophisticated reward signals and task complexity.

Reward Design and Mechanics

RoboNaldo's reward structure augments standard RL task signals with several innovations for credit assignment in high-impulse tasks:

• Motion Tracking Reward: Dense feedback encouraging adherence to human-derived reference states, providing stability early in training.
• Soccer-Specific Rewards: Comprising approach accuracy, high-impulse foot-ball contact, and final shot placement accuracy, these rewards are progressively weighted through the curriculum.
• Instant Interaction Reward: Recognizing the transient nature of soccer impact, this term accounts for approach geometry, impact magnitude, and resultant ball trajectory, providing dense, phase-specific supervision tailored to soccer shooting's dynamics.
• Densified Shooting Reward: Forward-extrapolates post-contact ball state to provide early and dense feedback on task outcomes, bridging the gap between short-duration control signals and delayed episode-level reward.

Simulation Evaluation

The framework is benchmarked against PPO, AMP, and state-of-the-art baselines such as PAiD in both stationary and moving ball shooting regimes. Key numerical results include:

• Stage 2 achieves 0.899 m mean free-kick shot error, outperforming prior SOTA by 48.6% and producing 2.96x higher shot velocity.
• Stage 3 generalizes to moving passes with 1.13 m mean shot error and 79.2% contact reliability across randomized trials, maintaining high velocity (13.88 m/s average).

Performance heatmaps illustrate spatial variations in accuracy, ball speed, and success rate relative to shot placement on the goal plane.

Figure 3: Target-conditioned heatmaps over goal-plane locations, visualizing shot error, peak ball speed, and success rates for both free-kick and moving-ball scenarios.

Real-World Deployment

RoboNaldo is deployed on a Unitree G1 (29-DOF, 35 kg) equipped with onboard LiDAR and camera perception. All inference and control are performed on-robot, and experiments span both indoor and unscripted outdoor environments.

Figure 4: Dispersion and trajectory plots for real-robot free-kick and moving-ball trials, including region-wise accuracy/speed metrics.

Strong numerical results include:

• 0.73 m average error from 3 m in free-kick settings (n=124), with 80.6% of shots landing within 1 m of the target and up to 13.10 m/s ball speeds (59-71% of professional player benchmarks).
• In moving-ball trials (n=20 valid), 0.86 m mean error and 70% within 1 m of target, despite increased variance due to perception and human pass timing.

Visually, the robot demonstrates stable, powerful, and targeted shooting across diverse configurations.

Figure 5: Time-lapse and snapshot sequences of outdoor free-kick and dynamic shooting, as well as landing outcome samples illustrating shot accuracy from various field locations.

Ablative Insights

Ablation studies concretize the necessity of each curriculum stage: omitting any stage or replacing the interaction reward with a sparser version significantly degrades both contact reliability and shot precision. Notably, removal of the stabilization phase or adaptive sampling results in a collapse of task success rate due to failures in post-impact balance and robustness.

Implications and Future Directions

RoboNaldo demonstrates that motion-guided curriculum RL can close the gap between kinematically stable, reward-driven control and the requirement for generalizable, high-impulse object interaction in humanoid robots. Its modular high-level interface allows for plug-in planning or co-training with policy-based decision modules, paving the way for flexible, multi-skill humanoid agents.

Future work should explore learning with multiple reference motions to support richer soccer skillsets, closing the gap to fully autonomous, decision-making soccer robots. The perception subsystem may also be extended to active egocentric vision without reliance on retro-reflective targets, aligning the system for deployment in unconstrained sporting environments.

Conclusion

This work presents a scalable approach for athletic humanoid interaction, coupling curriculum reinforcement learning with dense, phase-specific reward engineering and robust sim-to-real transfer. Comprehensive superiority in both simulation and real-world settings—across stability, accuracy, and power—underscore RoboNaldo's effectiveness as a foundation for next-generation athletic humanoid robots.