EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Data (2509.19626v1)

Abstract: Egocentric human experience data presents a vast resource for scaling up end-to-end imitation learning for robotic manipulation. However, significant domain gaps in visual appearance, sensor modalities, and kinematics between human and robot impede knowledge transfer. This paper presents EgoBridge, a unified co-training framework that explicitly aligns the policy latent spaces between human and robot data using domain adaptation. Through a measure of discrepancy on the joint policy latent features and actions based on Optimal Transport (OT), we learn observation representations that not only align between the human and robot domain but also preserve the action-relevant information critical for policy learning. EgoBridge achieves a significant absolute policy success rate improvement by 44% over human-augmented cross-embodiment baselines in three real-world single-arm and bimanual manipulation tasks. EgoBridge also generalizes to new objects, scenes, and tasks seen only in human data, where baselines fail entirely. Videos and additional information can be found at https://ego-bridge.github.io

• The paper proposes a domain adaptation framework for generalizable robotic imitation that leverages optimal transport guided by DTW.
• It aligns joint latent-action distributions from abundant human data with scarce robot data, resulting in improved simulation and real-world performance.
• Empirical results show significant gains in policy success rates and robust generalization to novel objects, scenes, and behaviors.

EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Data

Introduction and Motivation

EgoBridge addresses the challenge of transferring knowledge from large-scale egocentric human experience data to robotic manipulation policies. The core problem is the significant domain gap between human and robot data, encompassing differences in visual appearance, sensor modalities, and kinematics. While egocentric human data is abundant and diverse, robot data is expensive and limited. Existing approaches relying on naive co-training or marginal domain alignment fail to preserve action-relevant information, resulting in poor generalization and transfer. EgoBridge formalizes cross-embodiment imitation learning as a domain adaptation problem and introduces a joint distribution alignment framework using Optimal Transport (OT) guided by Dynamic Time Warping (DTW) on action trajectories.

Figure 1: EgoBridge enables knowledge transfer from human to robot by aligning latent representations using optimal transport, facilitating generalization to objects, scenes, and motions demonstrated only in human data.

Methodology

Joint Domain Adaptation via Optimal Transport

EgoBridge learns a shared latent space for both human and robot observations by explicitly aligning the joint distribution of latent features and actions. The feature encoder $f_\phi$ projects both human and robot observations into a common latent space $\mathcal{Z}$, and the policy $\pi_\theta$ maps these latents to actions. The key innovation is the use of a differentiable Sinkhorn OT loss on joint (latent, action) pairs, rather than marginal alignment. The OT cost matrix is shaped by DTW distances between human and robot action trajectories, enabling behaviorally meaningful pseudo-pairing.

The OT loss is defined as:

$$\mathcal{L}_{\text{OT-joint}}(\phi) = \sum_{i,j} (T^*_\epsilon)_{ij} \cdot \mathcal{C}\left( (f_\phi(o^H_i), a^H_i), (f_\phi(o^R_j), a^R_j) \right)$$

where $\mathcal{C}$ incorporates DTW-based behavioral similarity, strongly incentivizing alignment of robot samples with their closest human pseudo-pair.

Action-Aware Cost Function

DTW is used to compute the behavioral similarity between human and robot action sequences, robust to temporal and kinematic misalignments. The cost function discounts the latent distance for DTW-identified pseudo-pairs, focusing the OT alignment on semantically similar behaviors. This approach is shown to outperform naive MSE-based pairing and marginal OT alignment in both in-distribution and generalization settings.

Figure 2: Qualitative visualization of randomly sampled mini-batch pairing of MSE and DTW, demonstrating DTW's robustness to temporal and spatial misalignments.

Policy Architecture and Training

The policy network consists of modality-specific input stems (vision and proprioception), a shared transformer encoder trunk, and a multi-block transformer decoder. The OT-joint loss is applied to the encoder, while a standard behavior cloning loss supervises the entire network. Training is performed on mixed batches of human and robot data, with per-embodiment normalization and action chunking to improve temporal consistency.

Experimental Evaluation

Simulation: Push-T Benchmark

In a controlled planar pushing task, EgoBridge is evaluated against baselines including Maximum Mean Discrepancy (MMD), standard OT, naive co-training, and target-only training. The simulation probes both observation and behavior generalization, with diverse human data and narrow robot data.

Figure 3: Simulated Push-T experiments demonstrate EgoBridge's superior generalization to diverse human scenarios compared to domain adaptation baselines.

EgoBridge achieves the highest mean reward and success rate in both in-distribution and generalization settings, with a notably lower performance drop when tested on novel backgrounds and mirrored configurations.

Real-World Manipulation Tasks

Three real-world tasks are used for evaluation: Drawer (spatial generalization), Scoop Coffee (object and scene generalization), and Laundry (bimanual coordination). Human data covers broader scenarios than robot data, enabling tests of generalization to objects, scenes, and behaviors only seen in human demonstrations.

Figure 4: Distribution of human and robot training data, with evaluation settings highlighting in-distribution and out-of-distribution scenarios.

EgoBridge consistently outperforms baselines (Co-train, EgoMimic, MimicPlay, ATM, Robot-only BC) with up to 44% absolute improvement in policy success rate. Crucially, EgoBridge is the only method to generalize to novel objects, scenes, and behaviors, achieving nonzero success rates where all baselines fail.

Figure 5: Qualitative success of EgoBridge on real-world tasks, demonstrating robust transfer and generalization.

Figure 6: Common failure modes for each real-world evaluation task, highlighting the challenges addressed by EgoBridge.

Latent Space Analysis

T-SNE visualizations and KNN analysis of the learned latent space show that EgoBridge achieves the highest overlap and semantic alignment between human and robot data. Wasserstein-2 distance is minimized, and nearest neighbors correspond to similar task phases across embodiments.

Figure 7: T-SNE plots of encoded features for EgoBridge and baselines, with Wasserstein-2 distance and KNN pairs visualized.

Figure 8: Qualitative visualization of the learned latent space using T-SNE and K-Nearest Neighbors, showing semantic alignment of human and robot behaviors.

Ablation Studies

Ablations replacing DTW with MSE, joint OT with marginal OT, and removing alignment objectives demonstrate substantial drops in both in-distribution and generalization performance. The DTW-based cost is critical for identifying semantically similar pairs, and joint OT alignment is necessary for effective transfer.

Implementation Details

EgoBridge is implemented with modality-specific stems, a shared transformer trunk, and a DETR-style transformer decoder. Training uses AdamW, batch size 32, and GeomLoss for OT computation. Real-world experiments are conducted on a custom bimanual manipulator with egocentric and wrist-mounted cameras, and human data is collected using Project Aria smart glasses. Action chunking and per-embodiment normalization are applied for both human and robot data.

Figure 9: Aria glasses capture egocentric RGB images for both human and robot embodiments, with ground truth cartesian actions projected to pixel space.

Implications and Future Directions

EgoBridge demonstrates that joint distribution alignment using OT and DTW enables scalable and generalizable imitation learning from egocentric human data. The approach is robust to domain gaps and supports transfer to novel objects, scenes, and behaviors. The strong numerical results highlight the importance of action-aware alignment for cross-embodiment policy learning.

Limitations include the reliance on DTW for behavioral pairing, which may be less effective in multi-task or highly heterogeneous settings. Future work may explore alignment costs based on natural language embeddings or foundation model features, and extend the framework to multiple embodiments and unlabeled Internet-sourced human data.

Conclusion

EgoBridge introduces a principled domain adaptation framework for generalizable imitation learning from egocentric human data. By aligning joint latent-action distributions using OT guided by DTW, EgoBridge achieves significant improvements in policy success rates and robust generalization to novel scenarios. The method provides a scalable path for leveraging large-scale human experience data in robotic learning, with clear implications for future research in cross-embodiment transfer and foundation models for robotics.