Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations (2402.14606v1)

Published 22 Feb 2024 in cs.RO

Abstract: Imitation learning with human data has demonstrated remarkable success in teaching robots in a wide range of skills. However, the inherent diversity in human behavior leads to the emergence of multi-modal data distributions, thereby presenting a formidable challenge for existing imitation learning algorithms. Quantifying a model's capacity to capture and replicate this diversity effectively is still an open problem. In this work, we introduce simulation benchmark environments and the corresponding Datasets with Diverse human Demonstrations for Imitation Learning (D3IL), designed explicitly to evaluate a model's ability to learn multi-modal behavior. Our environments are designed to involve multiple sub-tasks that need to be solved, consider manipulation of multiple objects which increases the diversity of the behavior and can only be solved by policies that rely on closed loop sensory feedback. Other available datasets are missing at least one of these challenging properties. To address the challenge of diversity quantification, we introduce tractable metrics that provide valuable insights into a model's ability to acquire and reproduce diverse behaviors. These metrics offer a practical means to assess the robustness and versatility of imitation learning algorithms. Furthermore, we conduct a thorough evaluation of state-of-the-art methods on the proposed task suite. This evaluation serves as a benchmark for assessing their capability to learn diverse behaviors. Our findings shed light on the effectiveness of these methods in tackling the intricate problem of capturing and generalizing multi-modal human behaviors, offering a valuable reference for the design of future imitation learning algorithms.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Roboagent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking, 2023.
  2. Rt-1: Robotics transformer for real-world control at scale. In arXiv preprint arXiv:2212.06817, 2022.
  3. Specializing versatile skill libraries using local mixture of experts. In Conference on Robot Learning, pp.  1423–1433. PMLR, 2022.
  4. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint arXiv:2303.04137, 2023.
  5. Implicit behavioral cloning. In Conference on Robot Learning, pp.  158–168. PMLR, 2022.
  6. D4rl: Datasets for deep data-driven reinforcement learning. arXiv preprint arXiv:2004.07219, 2020.
  7. Benchmarks for deep off-policy evaluation. arXiv preprint arXiv:2103.16596, 2021.
  8. Arnold: A benchmark for language-grounded task learning with continuous states in realistic 3d scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
  9. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  18995–19012, 2022.
  10. Maniskill2: A unified benchmark for generalizable manipulation skills. In International Conference on Learning Representations, 2023.
  11. Rl unplugged: Benchmarks for offline reinforcement learning. arXiv preprint arXiv:2006.13888, 394, 2020.
  12. Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning. arXiv preprint arXiv:1910.11956, 2019.
  13. Furniturebench: Reproducible real-world benchmark for long-horizon complex manipulation. In Robotics: Science and Systems, 2023.
  14. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  15. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 5(2):3019–3026, 2020.
  16. Vima: General robot manipulation with multimodal prompts. In Fortieth International Conference on Machine Learning, 2023.
  17. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  18. One solution is not all you need: Few-shot extrapolation via structured maxent rl. Advances in Neural Information Processing Systems, 33:8198–8210, 2020.
  19. Infogail: Interpretable imitation learning from visual demonstrations. Advances in neural information processing systems, 30, 2017.
  20. Libero: Benchmarking knowledge transfer for lifelong robot learning, 2023.
  21. Learning latent plans from play. In Conference on robot learning, pp.  1113–1132. PMLR, 2020.
  22. Interactive language: Talking to robots in real time. IEEE Robotics and Automation Letters, 2023.
  23. Roboturk: A crowdsourcing platform for robotic skill learning through imitation. In Conference on Robot Learning, pp.  879–893. PMLR, 2018.
  24. What matters in learning from offline human demonstrations for robot manipulation. arXiv preprint arXiv:2108.03298, 2021.
  25. Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks. IEEE Robotics and Automation Letters, 7(3):7327–7334, 2022.
  26. Neural probabilistic motor primitives for humanoid control. arXiv preprint arXiv:1811.11711, 2018.
  27. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations. arXiv preprint arXiv:2107.14483, 2021.
  28. An algorithmic perspective on imitation learning. Foundations and Trends® in Robotics, 7(1-2):1–179, 2018.
  29. Hyperparameter selection for offline reinforcement learning. arXiv preprint arXiv:2007.09055, 2020.
  30. Imitating human behaviour with diffusion models. arXiv preprint arXiv:2301.10677, 2023.
  31. Goal-conditioned imitation learning using score-based diffusion policies. arXiv preprint arXiv:2304.02532, 2023.
  32. Behavior transformers: Cloning k𝑘kitalic_k modes with one stone. Advances in neural information processing systems, 35:22955–22968, 2022.
  33. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25, 2012.
  34. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28, 2015.
  35. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
  36. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pp.  5026–5033. IEEE, 2012.
  37. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  38. Bridgedata v2: A dataset for robot learning at scale. In 7th Annual Conference on Robot Learning, 2023. URL https://openreview.net/forum?id=f55MlAT1Lu.
  39. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on robot learning, pp.  1094–1100. PMLR, 2020.
  40. Transporter networks: Rearranging the visual world for robotic manipulation. In Conference on Robot Learning, pp.  726–747. PMLR, 2021.
  41. Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705, 2023.
Citations (19)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now