Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Polybot: Training One Policy Across Robots While Embracing Variability (2307.03719v1)

Published 7 Jul 2023 in cs.RO and cs.LG

Abstract: Reusing large datasets is crucial to scale vision-based robotic manipulators to everyday scenarios due to the high cost of collecting robotic datasets. However, robotic platforms possess varying control schemes, camera viewpoints, kinematic configurations, and end-effector morphologies, posing significant challenges when transferring manipulation skills from one platform to another. To tackle this problem, we propose a set of key design decisions to train a single policy for deployment on multiple robotic platforms. Our framework first aligns the observation and action spaces of our policy across embodiments via utilizing wrist cameras and a unified, but modular codebase. To bridge the remaining domain shift, we align our policy's internal representations across embodiments through contrastive learning. We evaluate our method on a dataset collected over 60 hours spanning 6 tasks and 3 robots with varying joint configurations and sizes: the WidowX 250S, the Franka Emika Panda, and the Sawyer. Our results demonstrate significant improvements in success rate and sample efficiency for our policy when using new task data collected on a different robot, validating our proposed design decisions. More details and videos can be found on our anonymized project website: https://sites.google.com/view/polybot-multirobot

View on arXiv
Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. doi:10.1109/CVPR.2009.5206848.
  2. Improving language understanding by generative pre-training. 2018.
  3. Robonet: Large-scale multi-robot learning. In Conference on Robot Learning, 2019.
  4. Learning modular neural network policies for multi-task and multi-robot transfer. In International Conference on Robotics and Automation, 2016.
  5. Hardware conditioned policies for multi-robot transfer learning, 2019.
  6. Know thyself: Transferable visual control policies through robot-awareness. In International Conference on Learning Representations, 2022.
  7. Bridging action space mismatch in learning from demonstrations. 2023.
  8. Multiple interactions made easy (mime): Large scale demonstrations data for imitation, 2018.
  9. Roboturk: A crowdsourcing platform for robotic skill learning through imitation. 2018.
  10. Visual imitation made easy. 2020.
  11. Bc-z: Zero-shot task generalization with robotic imitation learning. In Conference on Robot Learning, 2021.
  12. Bridge data: Boosting generalization of robotic skills with cross-domain datasets. In Robotics: Science and Systems, 2022.
  13. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on Robot Learning, 2019.
  14. Rlbench: The robot learning benchmark & learning environment. 2019.
  15. robosuite: A modular simulation framework and benchmark for robot learning. In arXiv preprint arXiv:2009.12293, 2020.
  16. D4rl: Datasets for deep data-driven reinforcement learning. 2021.
  17. Time-contrastive networks: Self-supervised learning from video. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 1134–1141, 2018. doi:10.1109/ICRA.2018.8462891.
  18. Real-world robot learning with masked visual pre-training. 2022.
  19. K. Grauman et al. Ego4d: Around the world in 3,000 hours of egocentric video, 2022.
  20. R3m: A universal visual representation for robot manipulation. In Conference on Robot Learning, 2022.
  21. Vip: Towards universal visual reward and representation via value-implicit pre-training. In International Conference on Learning Representations, 2023.
  22. Where are we in the search for an artificial visual cortex for embodied intelligence?, 2023.
  23. Language-driven representation learning for robotics. 2023.
  24. Learning generalizable robotic reward functions from "in-the-wild" human videos. 2021.
  25. Human-to-robot imitation in the wild. 2022.
  26. Concept2robot: Learning manipulation concepts from instructions and human demonstrations. 2020.
  27. R. Shah and V. Kumar. Rrl: Resnet as representation for reinforcement learning. 2021.
  28. Learning to see before learning to act: Visual pre-training for manipulation. 2021.
  29. Lossless adaptation of pretrained vision models for robotic manipulation. 2023.
  30. Vrl3: A data-driven framework for visual deep reinforcement learning. 2023.
  31. Semantic visual navigation by watching youtube videos. 2020.
  32. S. Reed et al. A generalist agent. Transactions on Machine Learning Research, 2022.
  33. Transfer from simulation to real world through learning deep inverse dynamics model. In CoRR, 2016.
  34. Preparing for the unknown: Learning a universal policy with online system identification. In CoRR, 2017.
  35. Sim-to-real transfer of robotic control with dynamics randomization. In International Conference on Robotics and Automation. IEEE, 2018.
  36. Sim2real view invariant visual servoing by recurrent control. In International Conference on Robotics and Automation, 2017.
  37. Gnm: A general navigation model to drive any robot. In International Conference on Robotics and Automation, 2023.
  38. Actor-mimic: Deep multitask and transfer reinforcement learning, 2016.
  39. One-shot imitation learning, 2017.
  40. One-shot visual imitation learning via meta-learning, 2017.
  41. Learning an embedding space for transferable robot skills. In International Conference on Learning Representations, 2019.
  42. Knowledge transfer in multi-task deep reinforcement learning for continuous control, 2020.
  43. Multi-task reinforcement learning with context-based representations. In International Conference on Machine Learning, 2021.
  44. A. Brohan et al. Rt-1: Robotics transformer for real-world control at scale, 2022.
  45. M. Ahn et al. Do as i can, not as i say: Grounding language in robotic affordances. 2022.
  46. Domain randomization for transferring deep neural networks from simulation to the real world. 2017.
  47. Virtual-to-real: Learning to control in visual semantic segmentation. In International Joint Conferences on Artificial Intelligence, 2018.
  48. Closing the sim-to-real loop: Adapting simulation randomization with real world experience. In 2019 International Conference on Robotics and Automation (ICRA), pages 8973–8979, 2019. doi:10.1109/ICRA.2019.8793789.
  49. Sim2real transfer for reinforcement learning without dynamics randomization, 2020.
  50. Retinagan: An object-aware approach to sim-to-real transfer, 2021.
  51. Using simulation and domain adaptation to improve efficiency of deep robotic grasping. 2017.
  52. Unpaired image-to-image translation using cycle-consistent adversarial networks. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2242–2251, 2017. doi:10.1109/ICCV.2017.244.
  53. Domain-adversarial training of neural networks, 2016.
  54. Multi-task domain adaptation for deep learning of instance grasping from simulation. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 3516–3523, 2018. doi:10.1109/ICRA.2018.8461041.
  55. Learning domain adaptation with model calibration for surgical report generation in robotic surgery. In International Conference on Robotics and Automation, 2021.
  56. Learning invariant feature spaces to transfer skills with reinforcement learning, 2017.
  57. Learning to correspond dynamical systems, 2020.
  58. Domain adaptation using system invariant dynamics models. In S. J. Wang and A. M. Johnson, editors, Proceedings of the 3rd Conference on Learning for Dynamics and Control, volume 144 of Proceedings of Machine Learning Research, pages 1130–1141. PMLR, 07 – 08 June 2021. URL https://proceedings.mlr.press/v144/wang21c.html.
  59. Vision-based manipulators need to also see from their hands, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Jonathan Yang (9 papers)
  2. Dorsa Sadigh (162 papers)
  3. Chelsea Finn (264 papers)
Citations (29)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now