PingpongDuo: Agile Humanoid Table Tennis Doubles with Safe Team Coordination
Abstract: Unlike singles, humanoid table tennis doubles requires teammates to strike in strict alternation while sharing a confined workspace. Rapid repositioning can compromise striking accuracy, while uncoordinated teammate movements introduce collision risks. We present PingpongDuo, a hierarchical framework for agile humanoid doubles with safe team coordination. At the team level, a planner jointly selects positioning targets and uses learned risk estimates to filter risky proposals, preferentially preserving the current striker's target. At the individual level, separate footwork and striking skills combine agile movement with precise racket control. The skills are guided by human motion references, including footwork collected from human doubles play, and trained from states produced by one another to support rapid transitions. A racket-arm residual further refines strikes after movement using return feedback from a simulator calibrated with real ball-flight and contact measurements. In simulation, PingpongDuo improves the ball-return success rate by 13.2% over the state-of-the-art baseline, while achieving approximately 2 cm positioning error during clearance. In real-world experiments, a robot completes a 2.15 m clearance-and-recovery cycle in 2.4 s with a peak speed of 2.23 m/s, while two robots sustain more than 10 consecutive strikes in strict alternation.
Paper Prompts
Sign up for free to create and run prompts on this paper.
Explain it Like I'm 14
1. What is the paper about?
This paper introduces PingpongDuo, a system that allows two humanoid robots to play table tennis doubles together.
Playing doubles is harder than playing alone. The robots must:
- Hit the ball one after another.
- Move quickly to the correct place.
- Get out of each other’s way.
- Avoid crashing into each other.
- Keep their balance while hitting a fast-moving ball.
The researchers designed a system that helps the robots plan together and move safely while still making accurate shots.
2. What questions did the researchers ask?
The paper mainly investigates three questions:
- Can two robots coordinate safely during a fast table-tennis rally? The robots must decide where both of them should stand and move.
- Is it better to train movement and hitting separately? For example, one skill could focus on running sideways, while another focuses on swinging the racket.
- Can the robot improve its hit after moving quickly? Movement may leave the robot slightly off-balance. The researchers wanted to know whether the robot could make a small last-second adjustment with its racket arm.
3. How did the researchers build and test the system?
Planning where the robots should stand
The system first predicts where the ball will go. It uses observations of the ball’s position and speed, together with a physics model that considers things such as gravity, air resistance, and bounces on the table.
Then, a team planner chooses a position for each robot:
- The robot that will hit the ball moves toward a good hitting position.
- The other robot moves away to create space.
- After the hit, the two robots switch roles.
The planner also uses a risk estimator. This is like a warning system that predicts whether a planned movement might cause the robots’ hands, feet, or bodies to come too close. If a movement looks dangerous, the system changes it to a safer one.
Importantly, it tries harder to preserve the hitting robot’s position. This is because moving the striker too much could make it miss the ball.
Teaching the robots how to move and hit
The researchers trained two main skills:
- Footwork skill: moving sideways, clearing space, waiting, and recovering.
- Striking skill: moving the body and racket to hit the ball accurately.
They collected about 200 examples of footwork from human table-tennis doubles players. These examples showed movements such as:
- Preparing to hit.
- Moving away after hitting.
- Recovering into position for the next turn.
The robot copied these human movements using a technique called motion retargeting. In simple terms, this means adapting a human movement so that it fits the robot’s body.
The researchers also trained the movement and striking skills to work together. A robot may begin hitting while it is still moving, or it may need to move immediately after a swing. The training therefore included these less-perfect transition situations instead of always starting from a neat, still position.
Improving the racket movement
The robot first uses a planned swing. Then a small extra controller, called a racket-arm residual, makes corrections to the racket arm.
This is similar to a student aiming at a target but making a tiny adjustment just before throwing. The correction depends on:
- The robot’s current position and movement.
- The ball’s position and speed.
- How well the robot is balanced.
The researchers trained this correction system in simulation. They first measured real ball flights, table bounces, and racket contacts so that the simulator behaved more like the real world. They also changed simulator conditions during training, such as ball weight and air resistance, so the robot would not only work in one perfect situation.
Testing the robots
The researchers tested PingpongDuo in:
- Computer simulations, where many trials could be run safely.
- Real-world experiments using two Unitree G1 humanoid robots.
The robots used external motion-capture cameras to track their bodies and the ball.
4. What did the researchers find?
Safer and better team coordination
In simulation, the complete system produced an average of 4.87 consecutive returns, compared with 3.79 for a simpler rule-based system.
The risk-aware system also avoided some collisions that occurred with other planning methods.
During clearance movements, the robots reached their target positions with about 2 centimeters of error. This means they moved very close to where they were supposed to go.
Better movement and striking when skills were separated
The researchers compared:
- One large policy that tried to learn movement and hitting together.
- Separate footwork and striking skills.
The separate-skill approach worked better. For PingpongDuo:
- The racket contacted the ball in 67.4% of attempts.
- The ball was legally returned in 55.6% of attempts.
Without the extra arm-refinement controller:
- Contact success fell to 55.2%.
- Legal-return success fell to 44.2%.
This shows that the small final racket correction was useful, especially when the robot was still moving or slightly unbalanced.
Successful real-world demonstrations
In real experiments:
- Two robots completed more than 10 alternating hits.
- One robot could clear space and recover over a distance of 2.15 meters in 2.4 seconds.
- The fastest measured movement reached 2.23 meters per second.
These results are important because they show that two humanoid robots can share a small space and play together instead of simply performing isolated single-player hits.
5. Why is this research important?
This paper shows that robots need more than the ability to hit a ball. In a team sport, they must also understand:
- Whose turn it is.
- Where their teammate is.
- How much space is needed.
- When to move quickly and when to slow down.
- How to stay safe while still completing the task.
The ideas in PingpongDuo could be useful beyond table tennis. Similar methods might help robots:
- Carry objects together.
- Work side by side in factories.
- Play or train in robot teams.
- Move safely around people or other robots.
- Coordinate during rescue or delivery tasks.
However, the system still has limitations. It depends on external motion-capture equipment, does not yet measure the ball’s spin, and cannot fully see the world using only sensors on the robots themselves.
Overall, the paper’s main message is that successful robot teamwork requires careful planning, safe movement, and specialized skills that cooperate with one another. PingpongDuo is an early demonstration of humanoid robots playing table-tennis doubles as a real team.
Practical Applications
Immediate Applications
- Multi-robot coordination in constrained workspaces — robotics and manufacturing
- Adapt the hierarchical planner, learned risk estimator, and role-weighted target filter to coordinate two or more robots operating near one another.
- Potential uses include robot handoffs, shared workbenches, warehouse picking, assembly cells, and collaborative inspection, where one robot must preserve task-critical positioning while another clears space or repositions.
- A deployable workflow could combine:
- 1. a high-level task or role assignment module,
- 2. learned collision-risk prediction,
- 3. constrained target adjustment, and
- 4. low-level motion skills.
- Dependencies: Reliable state estimation, calibrated robot geometry, conservative separation thresholds, and retraining or validation for each robot morphology and workspace. The paper’s safety results are demonstrated primarily in simulation, with limited real-world rally testing.
- Risk-aware motion planning and collision prevention — industrial robotics
- Use the learned risk filter as a safety layer around existing motion planners. Instead of replacing a production planner, the filter can reject or minimally modify target configurations predicted to produce hand, foot, or base collisions.
- This is particularly applicable to human–robot or robot–robot cells with dynamic role priorities, such as maintaining the active robot’s task while moving a secondary robot out of the way.
- The target-projection formulation provides a practical implementation pattern: preserve the higher-priority action and modify the lower-priority action whenever possible.
- Dependencies: The risk model must be trained on representative closed-loop rollouts; out-of-distribution motions, unexpected humans, actuator faults, and sensing delays may invalidate its predictions. A hard emergency-stop or model-based safety monitor should remain in place.
- Reusable agile footwork and manipulation skills — humanoid robotics
- The separation of
footwork,striking, clearance, recovery, and waiting policies can be reused for humanoid tasks involving frequent transitions between locomotion and manipulation. - Candidate applications include carrying objects while walking, dynamic tool use, mobile manipulation, kick-and-retrieve tasks, and rapid repositioning around workstations.
- Training each skill on states generated by the preceding skill offers a practical method for reducing failures at policy-switch boundaries.
- Dependencies: Human motion references must be retargetable to the target robot, and the robot must have sufficient balance, actuation, and whole-body control bandwidth. The reported performance depends on a 29-DoF humanoid and external motion capture.
- The separation of
- Residual control for dynamic contact tasks — robotics and automation
- The racket-arm residual can be generalized to tasks where a nominal motion is known but the final contact outcome depends strongly on small state errors.
- Examples include robotic batting, catching, throwing, pushing, insertion, tool contact, and dynamic grasping. A residual controller can modify only the task-critical joints while leaving the stable whole-body motion unchanged.
- The real-to-sim-to-real process—calibrating contact and environmental dynamics, randomizing uncertain parameters, and training with task-level outcome feedback—could support rapid adaptation without collecting extensive real-world trial data.
- Dependencies: A sufficiently accurate simulator and measurable task outcomes are required. Sim-to-real transfer may degrade for unmodeled friction, compliance, object deformation, spin, or sensor latency.
- Sports-training and entertainment systems — sports technology
- Deploy the system as a robotic table-tennis training partner, rally demonstrator, or coaching platform. The robot could maintain alternating rallies, vary landing locations, record player performance, and adapt its positioning difficulty.
- The ball-prediction and return-feedback components could also support automatic analysis of contact timing, racket orientation, movement efficiency, and recovery position.
- Dependencies: Human safety certification, protective operating zones, onboard perception, robust behavior around unpredictable human players, and support for ball spin are required. The current system relies on external motion capture and does not model spin.
- Academic research infrastructure — robotics and reinforcement learning
- The paper provides a concrete benchmark for studying:
- safe multi-agent reinforcement learning,
- hierarchical policy composition,
- sim-to-real transfer,
- contact-rich control,
- motion imitation,
- learned risk estimation, and
- multi-robot coordination under strict temporal role constraints.
- Researchers can reproduce the task with metrics such as legal return rate, alternating-rally length, clearance error, inter-robot separation violations, contact success, and landing accuracy.
- Dependencies: Access to humanoid hardware, motion-capture equipment, a physics simulator, synchronized ball tracking, and carefully defined safety protocols. Results should not be generalized from the reported five simulation seeds and limited real-world demonstrations without broader testing.
- Training-data and motion-library development — academia and robotics software
- The approximately 200 human doubles footwork clips can motivate datasets and software libraries for lateral shuffling, clearance, recovery, and movement-to-strike transitions.
- Similar pipelines could be used to collect human demonstrations for other cooperative tasks and retarget them to multiple robot embodiments.
- Dependencies: Licensing and privacy permissions for human motion data, high-quality segmentation, robust retargeting, and coverage of robot-specific joint limits and dynamics.
- Policy and safety evaluation for collaborative robots — standards and governance
- The paper’s role-weighted safety concept can inform evaluation procedures for systems in which safety constraints compete with task performance.
- For example, regulators or industrial safety teams could require reporting of collision-risk predictions, fallback behavior when no safe target exists, minimum separation margins, and the effect of safety filtering on task success.
- Dependencies: The learned risk score is not itself a formal safety guarantee. Certification would require interpretable bounds, systematic edge-case testing, hardware fault analysis, and likely a formal or verified safety layer.
Long-Term Applications
- General-purpose humanoid teamwork — robotics and logistics
- The framework could evolve into a general coordination architecture for teams of humanoids performing warehouse, construction, maintenance, or disaster-response tasks.
- Roles could dynamically alternate among carrying, clearing, observing, manipulating, and recovering, analogous to the striker/non-striker handoff in doubles.
- A future product could provide a reusable multi-humanoid coordination stack with task-level role assignment, risk-aware spatial negotiation, and skill-specific controllers.
- Dependencies: Extension from two robots and one-dimensional lateral targets to many robots, full three-dimensional planning, heterogeneous bodies, occlusion-aware perception, communication failures, and unpredictable human presence.
- Autonomous adversarial sports robots — sports robotics
- Future systems could select attacking and defensive strategies based on opponent behavior rather than merely returning incoming balls.
- This could produce autonomous practice opponents, competitive robot teams, and platforms for studying strategic multi-agent learning.
- Dependencies: Onboard ball and opponent perception, spin estimation, opponent-intent prediction, richer tactical planning, and significantly stronger safety guarantees. The paper explicitly identifies the absence of spin measurements and autonomous attack/defense selection as limitations.
- Onboard perception without motion capture — deployable humanoids
- Replacing external motion capture with cameras, depth sensors, event cameras, inertial sensors, and onboard ball tracking would make the system usable in ordinary environments.
- A future workflow could fuse visual ball tracking, robot state estimation, and learned uncertainty estimates before passing state information to the planner and risk filter.
- Dependencies: Low-latency perception at high frame rates, robust detection under occlusion and motion blur, accurate velocity estimation, synchronization across robots, and validation under changing illumination and backgrounds.
- Spin-aware and uncertainty-aware racket control — advanced sports and manipulation
- Incorporating ball spin, racket deformation, air-flow variation, and uncertainty in contact location could improve return placement and robustness.
- The residual controller could be extended to optimize distributions of likely landing points rather than a single predicted trajectory.
- Dependencies: Spin-capable sensing, richer aerodynamic and contact models, calibrated racket properties, and enough real-world data to validate the simulator. The current experiments do not establish performance under significant spin.
- Safe human–robot cooperation in shared spaces — healthcare, service robotics, and workplaces
- The role-prioritized risk filter could be adapted to situations where a robot must preserve a human’s intended motion while repositioning itself, or where two agents negotiate access to a shared workspace.
- Potential applications include hospital logistics, assisted rehabilitation, retail service, collaborative assembly, and domestic robots sharing narrow rooms.
- Dependencies: Human motion is less predictable than the scripted or simulated teammate behavior in the paper. Systems would need human-intent prediction, privacy-preserving sensing, explainable fallback behavior, and certified limits on speed and force.
- Transfer to other dynamic interaction domains — robotics, energy, and infrastructure
- The real-to-sim-to-real calibration method could be applied to contact-rich environments beyond sports, including robotic maintenance, valve operation, cable handling, docking, and manipulation of deformable or moving objects.
- The key transferable idea is to calibrate the dominant environmental interaction experimentally, then use domain randomization and bounded residual control to address remaining uncertainty.
- Dependencies: Each domain requires task-specific system identification and outcome measures. Transfer is not automatic because table-tennis contact is comparatively fast, lightweight, and repeatable.
- Large-scale multi-agent reinforcement-learning benchmarks — academia
- The task could become a benchmark for comparing constrained RL, control-barrier methods, temporal-logic planning, centralized training with decentralized execution, and risk-sensitive policy optimization.
- Future benchmark variants could include variable table layouts, more than two robots, communication delays, occluded observations, heterogeneous capabilities, and stochastic opponent strategies.
- Dependencies: Standardized environments, open-source implementations, reproducible hardware protocols, statistically meaningful evaluation, and explicit reporting of safety violations rather than only task rewards.
- Commercial humanoid sports and education products — consumer and education sectors
- A mature version could support physical-education classes, robotics curricula, public demonstrations, rehabilitation games, and interactive entertainment.
- The system could expose APIs for changing rally speed, target zones, movement difficulty, and safety limits, making it a controllable platform for teaching motor learning and robotics.
- Dependencies: Cost reduction, mechanical durability, low-maintenance sensing, safe interaction with children and non-experts, accessibility features, and evidence that performance remains reliable outside laboratory conditions.
Glossary
- Aerodynamic drag: Resistance exerted by air on a moving object, affecting its trajectory. “gravity, aerodynamic drag, and table bounces”
- Adaptive extended Kalman filter: A recursive estimator that combines noisy observations with a dynamic model to estimate changing system states. “An adaptive extended Kalman filter estimates ball position and velocity from successive observations.”
- Control Barrier Function (CBF): A mathematical safety mechanism that constrains system actions to keep states within a safe region. “Control Barrier Functions have also been explored for multi-robot coordination and safety.”
- Constrained policy optimization: A reinforcement-learning method that optimizes performance while restricting expected costs or constraint violations. “We train over sequences of incoming balls using constrained policy optimization based on PPO”
- Contact dynamics: The physical behavior governing interactions between bodies during impact, including forces, velocities, and restitution. “The identified dynamics define the nominal simulator”
- Contact-time error: The discrepancy between the simulated and measured times at which physical contact occurs. “The fitting objectives include time-aligned trajectory errors, rebound-height errors, and contact-time errors.”
- Domain randomization: Training with systematically varied simulated parameters so a policy becomes robust to modeling errors and real-world variation. “the refinement policy is trained with domain randomization over ball mass, friction, air drag, and restitution.”
- Extended Kalman filter: A nonlinear state-estimation algorithm that approximates system dynamics locally to infer hidden states from noisy measurements. “An adaptive extended Kalman filter estimates ball position and velocity from successive observations.”
- Free-flight dynamics: The motion of an object while it is traveling without contact with another surface or object. “We collect motion-capture trajectories covering racket--ball and table contacts to identify free-flight and contact dynamics.”
- GMR (Generalizable Motion Retargeting): A method for adapting human motion trajectories to the morphology and kinematic constraints of a robot. “We use GMR to retarget them to the robot”
- Heuristic CBF filtering: A manually designed safety-filtering approach based on heuristic rules and control barrier functions. “rule-based positioning, heuristic CBF filtering, learned-risk filtering”
- Hierarchical framework: A system organized into multiple decision-making levels, typically separating high-level planning from low-level control. “We present PingpongDuo, a hierarchical framework for agile humanoid doubles with safe team coordination.”
- Humanoid Jacobian: A matrix that relates changes in a humanoid robot’s joint variables to changes in the position and orientation of its body parts. “acquiring full observations and humanoid Jacobians in the real world is often infeasible.”
- Imitation learning: Learning a behavior by reproducing demonstrated motions or actions. “Training combines motion imitation, task rewards, and control regularization”
- Inter-robot separation: The distance between corresponding body parts or regions of different robots, used to assess collision risk. “The objective maximizes the accumulated team task return subject to a limit on the expected cost of inter-robot separation violations.”
- Joint policy: A single policy that learns to perform multiple behaviors or phases rather than using separate specialized policies. “Joint: one policy; Composed: separate footwork and striking skills”
- Learned risk estimator: A model trained from data to predict the likelihood of unsafe events for a proposed action or target. “A learned risk estimator supports a role-weighted target filter”
- Low-level controller: A control module that converts higher-level commands into detailed actuator or joint actions. “while holding the low-level controller and risk estimator fixed.”
- Motion capture (MoCap): A sensing technique that records the positions and movements of objects or body parts over time. “external motion capture (MoCap) for robot and ball state estimation.”
- Motion imitation: The process of training a controller to reproduce a demonstrated movement trajectory. “Its training combines position and velocity tracking, motion imitation, stability objectives, and control regularization.”
- Motion prior: A learned or recorded movement pattern that biases a system toward plausible actions. “providing motion priors for both short adjustments and larger lateral movements.”
- Motion retargeting: The adaptation of a motion performed by one body, such as a human, to a robot with different dimensions and joints. “We use GMR to retarget them to the robot”
- Multi-embodiment coordination: Coordination among robotic agents that may have multiple physical bodies or operate simultaneously in a shared environment. “Safe Multi-Embodiment Coordination”
- Nominal action: The baseline action generated by a primary controller before an additional corrective adjustment is applied. “The residual $\Delta a_{t,\mathrm{arm}^i$ is a bounded seven-dimensional correction for the racket arm.”
- Nominal simulator: The calibrated simulation model representing the expected physical system before randomized variations are introduced. “The fitted parameters define the nominal simulator”
- Policy residual: A learned corrective output added to the action of an existing policy or controller. “A racket-arm residual further refines strikes after movement”
- Proprioceptive observation: A robot’s internally sensed information about its own configuration and motion, such as joint positions and velocities. “The striking policy combines this command with proprioceptive observations and reference motion information”
- Reachability estimation: The computation of whether a robot can reach a state or location under its motion and control constraints. “ABS series integrate agile and recovery policies with reachability estimation”
- Reinforcement learning (RL): A learning paradigm in which an agent improves behavior through reward and penalty signals obtained from interaction with an environment. “safe RL approaches introduce cost functions and recovery policies”
- Residual body velocity: Remaining translational or rotational motion of a robot after a preceding movement has nominally ended. “Residual body velocity or an unsettled posture after footwork”
- Restitution: The physical property describing how much kinetic energy is retained after a collision, often represented through a coefficient of restitution. “table restitution, and friction parameters”
- Risk-aware target filter: A mechanism that modifies proposed target positions when their predicted safety risk exceeds an allowed threshold. “The RL planner is invoked per ball, using the predicted strike command to select both robots' positioning targets.”
- Racket-face normal: A unit vector perpendicular to the racket surface, used to characterize its orientation at contact. “to align its face normal with the desired strike direction”
- Racket–ball interaction model: A physical model that predicts how racket motion and ball motion combine during contact. “a racket--ball interaction model determines the required racket velocity.”
- Reference motion: A demonstrated or prerecorded trajectory used to guide a robot’s learned movement. “The striking policy combines this command with proprioceptive observations and reference motion information”
- Reinforcement-learning policy: A parameterized decision rule that maps observations to actions and is optimized using rewards or costs. “The RL planner is invoked per ball”
- Role-weighted safety filter: A safety mechanism that assigns different modification penalties to agents according to their current roles. “a role-weighted safety filter prioritizes the striker's target”
- State machine: A control architecture that switches among predefined behavioral modes according to system state and events. “During execution, a state machine combines the accepted positioning targets with the racket commands”
- State transition distribution: The distribution of physical states produced when one learned behavior hands control to another. “We therefore use states produced by each skill to initialize training of the other”
- Tangential attenuation: A reduction in the component of velocity parallel to the contact surface during an impact. “effective tangential attenuation.”
- Temporal gate: A time-dependent function that gradually activates or deactivates an action or correction during a specified phase. “A phase-dependent temporal gate gradually activates and removes the correction around contact.”
- Trajectory modeling: The mathematical representation and prediction of an object’s path through space and time. “Ball trajectory and racket--ball interaction modeling support accurate returns”
- Whole-body control: Coordinated control of many or all of a robot’s joints and body segments to achieve a task. “The reference is selected according to the strike target”



