Embodiment-Agnostic Representations

• Embodiment-agnostic representations are encodings that omit specific sensorimotor details, enabling cross-agent skill transfer between robots, humans, and cognitive models.
• They leverage methods like optical flow, latent skill vectors, and unified action spaces to achieve universal policy learning and improve sample efficiency.
• Empirical results show significant improvements in manipulation and navigation tasks, highlighting their potential for robust cross-domain generalization and abstract symbol grounding.

Embodiment-agnostic representations formalize the encoding of perception, action, skill, or concept in a manner independent of the specific morphology, kinematics, or sensorimotor structure of the agent. These representations intentionally abstract away embodiment details—such as joints, limbs, actuation specifics, or proprioceptive states—yielding generalizable signals that can be transferred, used, or interpreted across fundamentally different agents (e.g., between humans and robots, among distinct robotic morphologies, or even within cognitive models lacking any physical incarnation). Their principal utility spans cross-embodiment skill transfer, universal policy learning, unsupervised exploration, and grounded language understanding. Recent advances in robotics, reinforcement learning, navigation, and conceptual modeling have operationalized embodiment-agnostic representations using optical flow, scene/object-part 3D flow, latent skill vectors, unified action spaces, and abstract language-only structures, resulting in significant improvements in sample efficiency, generalization, and the fusion of heterogeneous datasets.

1. Formal Definitions and Theoretical Foundations

Embodiment-agnostic representations are initialized as encodings that omit explicit information about an agent's physical configuration. In cognitive science, this denotes representations learned or used without grounding in direct sensorimotor experience, a concept formalized in work comparing language-only large models (GPT-3.5) versus multimodal ones (GPT-4), where non-sensorimotor domains (emotion, salience, abstract imageability) exhibit high alignment with human judgments, while sensory and motor dimensions lag markedly (Xu et al., 2023).

In manipulation and control, an embodiment-agnostic representation may take the form of optic flow, which quantifies visual motion between consecutive observations:

$a_t = \mathrm{Flow}(o_t, o_{t+1}) \in \mathbb{R}^d$

Here, $a_t$ is a compact action surrogate immune to robot-specific actuation (Wang et al., 17 Jul 2025). For human-to-robot transfer, 3D object-part scene flow is defined as a tensor $s_{1:N} \in \mathbb{R}^{M \times 3 \times N}$, representing per-point displacements of manipulated object parts (Tang et al., 2024). More abstractly, cross-embodiment skill transfer is achieved via sparse optical-flow trajectories, with dynamic cues encoded as temporally consistent pixel displacements independent of morphology (Tang et al., 9 Oct 2025).

In theoretical models of symbol grounding, van Hateren shows representation and "aboutness" emerge when there exists an internal estimator $f_\mathrm{est}$ that reliably tracks an external Darwinian fitness $f_\mathrm{true}$, independent of embodiment (Hateren, 2015).

2. Model Architectures and Embodiment-Agnostic Learning Pipelines

Robotics and RL frameworks instantiate embodiment-agnostic world models and policies using specialized architectures. In manipulation, the 3DFlowAction pipeline leverages a video diffusion-based world model conditioned on: initial RGB frame, language prompt, and a sparse set of object 3D points. The output is a tensor of object-centric 3D flows, agnostic to the inducing agent's embodiment (Zhi et al., 6 Jun 2025). Optimization is then performed directly in 3D SE(3) pose space subject to predicted keypoint matches, producing action sequences that respect task dynamics but not robot-specific actuation details.

Embodiment-agnostic policy learning in PEAC proceeds by defining a Controlled-Embodiment MDP where the action space is a shared "action-embedding" and each embodiment has an associated projection $\phi_e:A \to A_e$, while policy pretraining is driven by an intrinsic reward maximizing cross-embodiment confusion (2405.14073). Similarly, Latent Action Diffusion learns contrastively aligned latent action spaces across human and robotic hands, enabling co-training of a single diffusion policy and subsequent decoding with embodiment-specific architectures (Bauer et al., 17 Jun 2025).

Navigation models such as ViDEN use depth images and relative target positions, encoded via CNN and MLP, to produce a transformer-based latent state $z_t \in \mathbb{R}^d$, which encodes the task in a way that is neutral to robot shape or kinematics. Policies are realized as conditional denoising diffusion models producing action waypoints in SE(2), rescaled to robot-specific velocity or pose bounds (Curtis et al., 2024). SwarmDiffusion removes robot-specific planner dependency: traversability and trajectory are predicted in parallel from RGB images and proprioceptive state, with universal conditioning ensuring cross-platform transfer (Zhura et al., 2 Dec 2025).

Table: Model Architectures for Embodiment-Agnostic Representations

Domain	Representation Type	Policy/WM Architecture
Manipulation	3D Scene/Object Flow	Video diffusion (U-Net) (Zhi et al., 6 Jun 2025)
RL Exploration	Action-embedding, skill space	CE-MDP encoder, diffusion, RL (2405.14073, Bauer et al., 17 Jun 2025)
Navigation	Depth image + relative pos.	CNN/MLP/transformer, diffusion (Curtis et al., 2024, Zhura et al., 2 Dec 2025)
Conceptual	Language-only embeddings	Distributional statistics (Xu et al., 2023)

3. Dataset Construction, Alignment, and Training Strategies

A major challenge in realizing embodiment-agnostic representations is harmonizing heterogeneous data sources. The ManiFlow-110k dataset (3DFlowAction) auto-detects and segments moving objects (grippers masked), tracks keypoints via Co-Tracker, extracts 2D flow, stabilizes camera motion, estimates depth by DepthAnything, and reconstructs object-centric flow instances suitable for cross-embodiment training (Zhi et al., 6 Jun 2025). UniSkill eliminates the need for paired data by pooling human and robot videos and training skill encoders (ISD) and editors (FSD) with latent diffusion objectives, ensuring skills represent only dynamic factors relevant to editing, not embodiment (Kim et al., 13 May 2025).

Alignment objectives vary: Latent Action Diffusion uses InfoNCE for contrastive alignment of retargeted actions and reconstruction losses for modality-specific decoders (Bauer et al., 17 Jun 2025). TrajSkill employs flow-magnitude weighted sampling for keypoint selection and sparse trajectory integration, anchoring skill transfer exclusively on movement cues rather than body shape (Tang et al., 9 Oct 2025). Dataset weighting, augmentation, and mixing techniques further control the balance and representation invariance across training domains.

4. Evaluation Metrics, Empirical Results, and Limitations

Embodiment-agnostic architectures are consistently validated using cross-embodiment generalization, sample efficiency, and success rates on both simulation and real-world tasks:

• 3DFlowAction achieves 70% in-domain manipulation success versus 20–25% for prior models, and up to 70% zero-shot transfer between Franka and XTrainer robots; 55% success on unseen objects (Zhi et al., 6 Jun 2025).
• PEAC shows normalized IQM score of 0.69 versus 0.62 (baseline), and retains a 15–30% edge in zero-shot generalization to unseen robots (2405.14073).
• SwarmDiffusion maintains 80–100% navigation success with only ~500 new samples for adaptation, and inference times of 0.09 s; ablations demonstrate traversability and FiLM conditioning are essential (Zhura et al., 2 Dec 2025).
• UniSkill achieves up to 91% (LIBERO robot-only prompts) and 48% (human prompts) success, surpassing alignment baselines by 2–4× (Kim et al., 13 May 2025).
• TrajSkill’s sparse-flow representation yields up to 16.7% improvement in cross-embodiment MetaWorld success rates (Tang et al., 9 Oct 2025).

Limitations observed include degraded performance on non-rigid or occluded objects (3DFlowAction), sensitivity to viewpoint variation (UniSkill), loss of fine-grained content in pure language-based models (GPT-3.5 in (Xu et al., 2023)), and challenges in scaling contrastive latent spaces to very heterogeneous or asymmetrically observed robots (Bauer et al., 17 Jun 2025).

5. Connections to Symbol Grounding and Conceptual Representation

Embodiment-agnostic representation is situated in a broader theoretical context as a means to learn concepts, skills, and symbols in ways not tethered to physical experience. Xu et al. demonstrate that linguistic distributional learning is sufficient for non-sensorimotor domains, while sensorimotor-rich representation demands additional embodied data (Xu et al., 2023). Van Hateren's model encodes aboutness via the relation between internal estimator $f_\mathrm{est}$ and actual fitness $f_\mathrm{true}$, underscoring that genuine symbol grounding may require agency and nonlinearity mechanics more generally absent in most artificial agents (Hateren, 2015). A plausible implication is that embodiment-agnostic architectures can scaffold high-level reasoning or universal policies, but ultimate referential meaning for survival goals may only be attained through embodiment-sensitive learning dynamics.

6. Future Directions and Open Challenges

Key directions suggested across works include extending rigidity assumptions to non-rigid or articulated object flows (cloth, rope), integrating multi-view and tactile data for improved depth/occlusion handling, scaling datasets and architectures to cover more diverse robots and environmental contexts, and developing curriculum learning to incrementally enrich modality alignment (Zhi et al., 6 Jun 2025, Xu et al., 2023). The challenge of aligning policy or conceptual spaces across radically different bodies involves continued exploration of invariance/equivariance principles (Chen et al., 18 Sep 2025), novel skill discovery methods (Kim et al., 13 May 2025), and theoretical advances in symbol grounding beyond reproduction-driven agency (Hateren, 2015).

In summary, embodiment-agnostic representation serves as a foundational abstraction in both robotics and cognitive modeling, enabling scalable, transferable, and generalizable learning. Its further refinement and operationalization are critical for the emergence of universal policies, robust skill transfer, cross-domain concept formation, and ultimately the realization of autonomous agents capable of reasoning, acting, and communicating irrespective of their physical form.