Cross-Embodiment Robot Manipulation via a Unified Hand Action Space

Abstract: Robot manipulation policies are typically tied to specific robotic hand embodiments, limiting the transfer of learned behaviors across platforms with different kinematic structures. In this work, we propose the Unified Hand Action Space (UHAS), a sphere-based unified action representation for cross-embodiment dexterous manipulation. UHAS represents robotic hand actions as geometric deformations of a canonical sphere and uses a Cascade Inverse Kinematics (CIK) algorithm to map the shared representation to embodiment-specific joint configurations. Using reinforcement learning, we train dexterous manipulation policies directly in the proposed action space for in-hand cube reorientation tasks. We evaluate our method in both simulation and real-world experiments across multiple robotic hands, including the Allegro Hand, LEAP Hand, Shadow Hand, and MANO Human Hand. Experimental results demonstrate effective dexterous manipulation, zero-shot transfer to unseen hands, rapid finetuning across embodiments, and successful real-world deployment. Our experiments show that the proposed UHAS representation enables stable dexterous control and cross-embodiment policy transfer across robotic hands.

• The paper introduces a geometric, sphere-based Unified Hand Action Space (UHAS) that facilitates policy transfer across diverse robotic hands.
• It employs a Cascade Inverse Kinematics (CIK) algorithm to rapidly convert sphere deformations into hand-specific joint commands for dexterous manipulation.
• Experimental results demonstrate significant zero-shot transfer, fast adaptation, and superior performance over traditional joint-space baselines in cube reorientation tasks.

Cross-Embodiment Robot Manipulation via a Unified Hand Action Space

Introduction

The paper "Cross-Embodiment Robot Manipulation via a Unified Hand Action Space" (2607.03570) presents a robust geometric framework to address the long-standing challenge of policy and skill transfer across dexterous robotic hands with divergent kinematic structures. The core innovation is the Unified Hand Action Space (UHAS), which parameterizes manipulation actions as deformations of a canonical sphere rather than as joint torques or positions. UHAS is coupled with a Cascade Inverse Kinematics (CIK) algorithm that resolves embodiment-specific joint commands from this geometric action specification. The framework is validated with reinforcement learning agents trained to perform in-hand cube reorientation across a set of robot hands with varying morphologies, including Allegro, LEAP, Shadow, and MANO, demonstrating substantial gains in cross-embodiment transfer, zero-shot generalization, and rapid adaptation.

Figure 1: In the unified hand action space, an action is a deformation of a canonical sphere which is mapped to embodiment-specific hand configurations (LEAP, Allegro, MANO, Shadow).

Sphere-Based Unified Action Representation

Traditional end-effector action spaces (e.g., joint torques, Cartesian poses) severely limit direct manipulation skill transfer due to hardware-specific constraints. UHAS instead encodes actions as spatial deformations on a canonical, normalized sphere embedded within the grasp workspace of any given hand.

Figure 2: Construction of the canonical sphere for a robotic hand in URDF, centered and oriented by palm and fingertip geometry.

This design delivers multiple invariances:

1. Kinematic Structure Invariance: The sphere and its parameterization (center, radius, axes) are automatically derived via geometric normalization from any hand's URDF, ensuring the encoding is consistent across varying finger layouts.
2. Scale Invariance: All coordinates are normalized by the reference radius, yielding a unit sphere.
3. Surface Correspondence: Surface points on the sphere are densely mapped through projection onto corresponding points on each hand's palm or fingers, maintaining configuration invariance during manipulation.

   Figure 3: Workflow for creating unified hand surface correspondences between the sphere and arbitrary hand geometries.

Sphere Deformation Parameterization

To sidestep the impracticality of a fully dense action specification, UHAS employs a compact, yet expressive, sphere deformation parameterization. Deformations are described by azimuthal and radial changes $(\Delta\theta, \Delta r)$, using a finite set of driving planes (one per finger) and discrete radial control points per plane ("driving vectors").

Figure 4: Parameterization and interpolation of sphere deformations using a set of azimuthal driving planes and control vectors.

The geometric semantics are:

• Driving Plane Rotation ($\Delta\theta$): Modulates lateral sweeps.
• Radial Displacement ($\Delta r$): Modulates finger closing or open distance. This leads to a low-dimensional, hardware-agnostic action vector suitable for high-dimensional RL.

Cascade Inverse Kinematics (CIK)

To exploit UHAS for physical robot control, the paper introduces the CIK algorithm, which robustly inverts the geometric deformation mapping into hand-specific joint positions, in real-time.

Each hand's joints are classified (once per URDF) into:

• Lateral joints: Primarily control the fingertip azimuthal angle (side-to-side sweeping).
• Encompassing joints: Control closing motion and shape conformity.

CIK then sequentially resolves target surface positions, first by direct lookup for lateral joints (using precomputed $\theta$ tables), followed by a cascade solving the encompassing joints via fast one-dimensional IK, ensuring physically plausible actuation throughout the kinematic chain.

Figure 5: (a) Classification of lateral vs. (b) encompassing joints; (c) CIK applied on a deformed sphere for hand motion synthesis.

Experimental Evaluation

Simulation Suite

In-hand cube reorientation is used as the primary benchmark, evaluated on Allegro, LEAP, Shadow, and MANO hands, with both four- and five-fingered topologies. Metrics include average number of consecutive successful reorientations and per-target success rate. RL agents are implemented via PPO (RSL-RL) and conditioned on a homogeneous, embodiment-agnostic observation space built from canonical sphere correspondences.

Figure 6: (a) Simulation environments with four alternative hand morphologies; (b) real-world setup of the LEAP hand.

Findings:

• Single-Hand and Multi-Hand Training: UHAS trained on the union of all hands matches or exceeds policies trained on each hand individually and outperforms joint-space baseline controllers.
• Zero-Shot Transfer: When trained on a subset of hands and deployed on an unseen hand, UHAS achieves notably high zero-shot transfer performance (success rates up to 95% on some hands; see [Table in paper]), even across differing finger counts and kinematics.
• Fast Finetuning: Policies transferred to new hands can be adapted within a small number of RL iterations, a key result for practical deployment.

Cross-Morphology Results

UHAS supports effective transfer even between qualitatively different hand structures (e.g., 4-finger to 5-finger). While policies perform best in-distribution, out-of-distribution generalization is strong and success rates substantially exceed baseline methods.

Real-World Deployment

On physical hardware (LEAP, Allegro hand), sim-to-real transfer is validated. Despite the evident reality gap (imperfect system identification, sensor noise, lower-fidelity controllers), UHAS policies—particularly when finetuned—substantially outperform joint-space baselines and can execute non-trivial, multi-stage cube reorientation with minimal per-hand retraining.

Figure 7: Example step in a real-world run of UHAS policy for in-hand cube reorientation on LEAP hardware.

System and Algorithmic Insights

• CIK executes at >100Hz on commodity hardware; control bottlenecks stem from sensor and motor comms, not UHAS overhead.
• Embodiment-agnostic observation encoding (canonical sphere-relative positions/velocities) is essential for transfer.
• Compact action spaces (2 driving vectors per plane) suffice; increasing action dimensionality yields diminishing returns.
• Domain randomization and robust reward design are necessary for sim-to-real generalization but are not a substitute for geometric abstraction.

  Figure 8: Homogeneous, hand-agnostic observation points (midpoint, fingertip) for LEAP, Allegro, Shadow, MANO.

Implications and Future Directions

This work provides a scalable solution for cross-embodiment, multi-morphology dexterous manipulation policy synthesis—enabling modular policies that are trained once and deployed on heterogeneous platforms with little or no retraining. Theoretically, it validates geometric abstraction as a key enabler for agent generalization in the context of high-DOF underactuated hands. Practically, it reduces the requirements for per-robot engineering and opens the path for generalist dexterous foundation models.

There are natural next steps:

• Extension to tasks involving dynamic contact, non-prehensile manipulation, and tool use.
• More expressive geometric parameterizations or implicit-function-based action spaces to capture even richer behaviors.
• Integration with vision-language-action policy models for semantically conditioned, cross-embodiment manipulation.
• Policy distillation or modular systems that can automatically grow, prune, or adapt action representations for novel kinematics.

Conclusion

UHAS and CIK together establish a unified, efficient, and transferable framework for learning and deploying dexterous manipulation policies on a wide array of robot hands. Action parameterization via sphere deformations abstracts away disparities in actuation and morphology, facilitating cross-embodiment generalization, fast adaptation, and significant improvements over joint-space or hand-specific control. While open challenges remain in sim-to-real transfer and out-of-distribution morphologies, this work substantially strengthens the case for geometric action representations as a universal interface for dexterous robotic manipulation.