SPACE: Enabling Learning from Cross-Robot Data Toward Generalist Policies

Abstract: In robot learning, scaling training datasets across diverse embodiments and environments has become a dominant paradigm for learning generalizable robot policies. These policies are commonly trained via behavior cloning to imitate actions from pre-collected demonstrations. However, since robot actions are tied to the dynamics of the data collection robot, different robots may require different actions to achieve the same motion. This discrepancy hinders both policy training and deployment across diverse robots. To address this, we propose using Cartesian state delta as a universal action representation across robots, and introduce State Prediction and Adaptive Command Execution (SPACE) framework. SPACE handles robot dynamics variation at three levels: across different embodiments, across hardware units of the same embodiment, and within a single robot during operation. It consists of two components: (i) a Cartesian state delta policy that predicts geometric end-effector displacement, and (ii) Action Adapter, which converts the predicted Cartesian state delta into robot-specific control commands. Experiments show that SPACE substantially outperforms policies that directly predict control commands when learning from data collected across different embodiments and across hardware units of the same embodiment. SPACE also remains robust under dynamics shifts at deployment, including changes in control frequency, object weight, and controller gains. The project page is available at http://haeone.site/space-website/.

• The paper introduces SPACE, which decouples robot-specific commands by predicting Cartesian state deltas to address inconsistencies across diverse robotic systems.
• It employs a linear Action Adapter with least-squares calibration and online LMS updates to robustly adapt to dynamic execution and intra-robot changes.
• Experimental results demonstrate up to 92% zero-shot success and significant error reduction, outperforming traditional command-based approaches across multiple platforms.

Enabling Cross-Robot Policy Generalization via Cartesian State Delta and Adaptive Command Execution

Problem Motivation and Context

Robotic policy learning at scale demands leveraging heterogeneous datasets—across robot embodiments, hardware instances, and demonstration modalities. However, robot control commands are inherently non-portable: the same command may induce disparate physical trajectories on robots differing in kinematics, low-level controllers, wear state, and other dynamics factors. This inconsistency severely hinders transfer and generalization for policies trained from such data, limiting the utility of large-scale multi-robot datasets. Moreover, many demonstration sources (e.g., kinesthetic teaching, hand-held grippers) lack explicit control command records, further challenging unified action representation design.

Recent research efforts acknowledge the challenge but typically address it by either designing embodiment-specific action heads, incorporating embodiment information, or learning latent actions—a strategy not fundamentally addressing the cross-robot inconsistency of control commands, and thus requiring robot-specific finetuning. Approaches like domain randomization or offline gain tuning further demand either access to simulation or extensive parameter searches, constraining practical scaling.

SPACE: Framework Construction

The SPACE (State Prediction and Adaptive Command Execution) framework presents a unified, controller-independent approach founded on two mechanisms: Cartesian state delta policy learning and an Action Adapter module.

Cartesian State Delta Policy

Instead of predicting robot-specific control commands, policies trained under SPACE predict end-effector displacements—i.e., the delta in the end-effector’s pose (translation and orientation) between consecutive timesteps in Cartesian space. This representation is invariant to an individual robot’s kinematics, base frame, and control interface, requiring only access to end-effector pose history. During training, the policy maximizes the likelihood of these deltas from demonstration trajectories, decoupling the action labeling process from the specifics of low-level robot control.

Action Adapter

To bridge the gap between the geometry-centric policy outputs and robot-specific command execution, the Action Adapter parameterizes the mapping from predicted Cartesian state deltas to target robot commands as a linear function, $u = W\Delta p + b$. Initial parameters are fitted via least-squares regression from a brief (∼1 minute) robot-specific random calibration procedure. To maintain robust adaptation during deployment under time-varying dynamics (e.g., changing payload, controller gains, frequency shifts), the Adapter parameters are continuously updated online using a least mean squares (LMS) procedure, directly leveraging realized end-effector motion for error correction.

Experimental Results

Cross-Embodiment Generalization

SPACE demonstrates a substantial advantage when polices are co-trained on data from different robot embodiments and then evaluated in zero-shot and co-training scenarios. In transfers between UR5 and Franka Research 3 (FR3) robots, as well as combinations involving human hand-held gripper data (FastUMI), SPACE achieves up to 92% zero-shot success rate as opposed to 0% for control-command-predicting baselines, and an average 30–50% success rate improvement in co-training. Critically, direct replay of UR5/UMI control commands on an FR3 led to severe tracking failures (∼100mm errors), while the SPACE strategy reduced these errors by over 85%.

Cross-Hardware Robustness

When trained on one hardware instance and evaluated on another ostensibly identical unit, traditional action-space policies degrade precipitously: success rates on the “PnP Box” task fell from 98% to 18%. In contrast, SPACE maintained a robust performance (84% success rate), verifying that the policy is decoupled from hardware-specific idiosyncrasies and can absorb unit-to-unit dynamical variation.

SPACE also scales efficiently to multi-hardware datasets (DROID). When fine-tuned and deployed on two diverse manipulation tasks, it delivers 80–84% higher success rates than competing methods based on direct control commands, joint velocities, or positions.

Adaptation to Intra-Robot Dynamics Shifts

The adaptive online updating of the Action Adapter enables policies learned using SPACE to rapidly compensate for changes in execution conditions. Neither increased control Hz nor dramatic variations in payload (increasing box weight from 90g to 530g) led to significant drops in performance for SPACE (92–96% success, comparable to original conditions); meanwhile, baseline approaches suffered catastrophic failures (0–50% success) under the same perturbations. Moreover, modifying controller gains (by 0.5× or 1.5× compared to training values) degraded traditional action-based policies to 0–25% success, while SPACE consistently preserved high performance.

Ablation and Baseline Comparisons

Ablation studies demonstrate that both offline initialization and online LMS updating are necessary for the Action Adapter to achieve robust dynamics adaptation. Alternatives such as gain tuning or delta accumulation, while occasionally competitive on paper, fell short in practical replay and execution experiments, with tracking errors and success rates lagging SPACE by large margins.

Implications and Future Directions

Practical Impact

SPACE provides a minimally invasive, scalable, and dynamics-agnostic layer for action abstraction, making it possible to leverage large-scale cross-embodiment and cross-hardware datasets with negligible per-robot calibration and no need for access to low-level control details. This substantially lowers the engineering and data collection barriers to developing “generalist” robotic policies robust to the real-world idiosyncrasies of hardware.

Limitations and Open Challenges

• Force/Compliance Tasks: Cartesian state delta does not directly encode the force or compliance constraints required for certain manipulation tasks. Future research may explore augmenting the action space to explicitly model or predict force/torque objectives in addition to displacement.
• Non-Cartesian Command Interfaces: The current Action Adapter assumes the target controller accepts Cartesian delta commands. Generalizing Adapter logic to robots exposing only joint-space or more constrained command APIs remains an open technical challenge.

Broad Theoretical Directions

The SPACE method highlights that geometric invariants, paired with adaptive mapping layers, can serve as effective “universal” action representations across embodiments and time-varying conditions—an insight likely to shape future robotic foundation models and large-scale multi-robot collaborative learning paradigms. Its modular integration with existing VLA models suggests that robust, scalable policy learning is tractable even in the presence of the heterogeneity and instability endemic to large-scale real-world robotics data.

Conclusion

The SPACE framework establishes a compelling mechanism for generalist robot policy learning by decoupling action representation from robot-specific dynamics through Cartesian state delta prediction and adaptive action command mapping. Experimental evaluations validate its superiority over traditional command-centric approaches in cross-embodiment, multi-hardware, and nonstationary settings. There remains substantial scope for extension in incorporating force/control abstraction and expanding Adapter flexibility, but SPACE offers a practical path toward reliable, transferable, and scalable robot imitation learning from diverse data sources (2606.24049).