Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

EVE: Enabling Anyone to Train Robots using Augmented Reality (2404.06089v3)

Published 9 Apr 2024 in cs.HC and cs.RO

Abstract: The increasing affordability of robot hardware is accelerating the integration of robots into everyday activities. However, training a robot to automate a task requires expensive trajectory data where a trained human annotator moves a physical robot to train it. Consequently, only those with access to robots produce demonstrations to train robots. In this work, we remove this restriction with EVE, an iOS app that enables everyday users to train robots using intuitive augmented reality visualizations, without needing a physical robot. With EVE, users can collect demonstrations by specifying waypoints with their hands, visually inspecting the environment for obstacles, modifying existing waypoints, and verifying collected trajectories. In a user study (N=14, D=30) consisting of three common tabletop tasks, EVE outperformed three state-of-the-art interfaces in success rate and was comparable to kinesthetic teaching-physically moving a physical robot-in completion time, usability, motion intent communication, enjoyment, and preference (mean of p=0.30). EVE allows users to train robots for personalized tasks, such as sorting desk supplies, organizing ingredients, or setting up board games. We conclude by enumerating limitations and design considerations for future AR-based demonstration collection systems for robotics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. 2023. MoveIt Motion Planning Framework. https://moveit.picknik.ai/main/doc/concepts/motion_planning.html. Accessed: 2023-04-03.
  2. Barış Akgün and Kaushik Subramanian. 2011. Robot Learning from Demonstration : Kinesthetic Teaching vs . Teleoperation. https://api.semanticscholar.org/CorpusID:45150299
  3. A survey of robot learning from demonstration. Robotics and Autonomous Systems 57, 5 (2009), 469–483.
  4. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817 (2022).
  5. John Brooke. 1996. SUS: A ”quick and dirty” usability scale. Usability evaluation in industry 189, 3 (1996), 189–194.
  6. Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots. arXiv preprint arXiv:2402.10329 (2024).
  7. Designing persuasive robots: how robots might persuade people using vocal and nonverbal cues. In Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction (Boston, Massachusetts, USA) (HRI ’12). Association for Computing Machinery, New York, NY, USA, 293–300. https://doi.org/10.1145/2157689.2157798
  8. Legibility and predictability of robot motion. 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (2013), 301–308. https://api.semanticscholar.org/CorpusID:15202367
  9. AR2-D2: Training a Robot Without a Robot. (2023).
  10. Hugh Durrant-Whyte and Tim Bailey. 2006. Simultaneous Localisation and Mapping (SLAM): Part I The Essential Algorithms. IEEE Robotics & Automation Magazine 13, 2 (2006), 99–110. https://doi.org/10.1109/MRA.2006.1638022
  11. Bridge data: Boosting generalization of robotic skills with cross-domain datasets. arXiv preprint arXiv:2109.13396 (2021).
  12. Elephant Robotics. 2023. mechArm Pi: The Most Compact 6-Axis Robot Arm. https://shop.elephantrobotics.com/collections/mecharm/products/mecharm. Accessed on [insert date of access].
  13. A Comparison of Types of Robot Control for Programming by Demonstration. In The Eleventh ACM/IEEE International Conference on Human Robot Interaction (Christchurch, New Zealand) (HRI ’16). IEEE Press, 213–220.
  14. Chien-Ming Huang and Bilge Mutlu. 2013. Modeling and Evaluating Narrative Gestures for Humanlike Robots. In Robotics: Science and Systems. https://api.semanticscholar.org/CorpusID:13278469
  15. Characterizing the Design Space of Rendered Robot Faces. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (Chicago, IL, USA) (HRI ’18). Association for Computing Machinery, New York, NY, USA, 96–104. https://doi.org/10.1145/3171221.3171286
  16. EchoBot: Facilitating data collection for robot learning with the Amazon echo. In 2017 13th IEEE Conference on Automation Science and Engineering (CASE). IEEE, 159–165.
  17. BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation. https://api.semanticscholar.org/CorpusID:268520018
  18. Planning Safe and Legible Hand-over Motions for Human-Robot Interaction. https://api.semanticscholar.org/CorpusID:6846660
  19. Roboturk: A crowdsourcing platform for robotic skill learning through imitation. In Conference on Robot Learning. PMLR, 879–893.
  20. A Storytelling Robot: Modeling and Evaluation of Human-like Gaze Behavior. 2006 6th IEEE-RAS International Conference on Humanoid Robots (2006), 518–523. https://api.semanticscholar.org/CorpusID:18092365
  21. Crowdsourcing for closed loop control. In Proc. of the NIPS Workshop on Computational Social Science and the Wisdom of Crowds. NIPS, 4–7.
  22. Hello Robot. 2023. Hello Robot: Open Source Mobile Manipulator for AI & Robotics. https://hello-robot.com/. Accessed on [insert date of access].
  23. Communicating Robot Arm Motion Intent Through Mixed Reality Head-mounted Displays. arXiv:1708.03655 [cs.RO]
  24. HCI Guidelines for Gender Equity and Inclusivity. In UMBC Faculty Collection. UMBC.
  25. Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperation. In 2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids). IEEE, 1–8.
  26. Leap hand: Low-cost, efficient, and anthropomorphic hand for robot learning. arXiv preprint arXiv:2309.06440 (2023).
  27. Videodex: Learning dexterity from internet videos. In Conference on Robot Learning. PMLR, 654–665.
  28. Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation. arXiv:2209.05451 [cs.RO]
  29. Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 553, 33 pages. https://doi.org/10.1145/3491102.3517719
  30. Daniel Szafir and Danielle Albers Szafir. 2021. Connecting human-robot interaction and data visualization. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction. 281–292.
  31. A force-sensitive exoskeleton for teleoperation: An application in elderly care robotics. In 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 12624–12630.
  32. Communicating Robot Motion Intent with Augmented Reality. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (Chicago, IL, USA) (HRI ’18). Association for Computing Machinery, New York, NY, USA, 316–324. https://doi.org/10.1145/3171221.3171253
  33. MimicPlay: Long-Horizon Imitation Learning by Watching Human Play. arXiv:2302.12422 [cs.RO]
  34. DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation. arXiv preprint arXiv:2403.07788 (2024).
  35. A user study on kinesthetic teaching of redundant robots in task and configuration space. Journal of Human-Robot Interaction 2, 1 (2013), 56–81.
  36. Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators. arXiv preprint arXiv:2309.13037 (2023).
  37. A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges. arXiv:2309.02473 [cs.LG]
  38. Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705 (2023).
  39. Viola: Imitation learning for vision-based manipulation with object proposal priors. arXiv preprint arXiv:2210.11339 (2022).
  40. FlyAR: Augmented Reality Supported Micro Aerial Vehicle Navigation. IEEE Transactions on Visualization and Computer Graphics 20, 4 (2014), 560–568. https://doi.org/10.1109/TVCG.2014.24
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Jun Wang (991 papers)
  2. Chun-Cheng Chang (3 papers)
  3. Jiafei Duan (26 papers)
  4. Dieter Fox (201 papers)
  5. Ranjay Krishna (116 papers)
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets