Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SENSOR: Imitate Third-Person Expert's Behaviors via Active Sensoring (2404.03386v1)

Published 4 Apr 2024 in cs.RO, cs.AI, and cs.LG

Abstract: In many real-world visual Imitation Learning (IL) scenarios, there is a misalignment between the agent's and the expert's perspectives, which might lead to the failure of imitation. Previous methods have generally solved this problem by domain alignment, which incurs extra computation and storage costs, and these methods fail to handle the \textit{hard cases} where the viewpoint gap is too large. To alleviate the above problems, we introduce active sensoring in the visual IL setting and propose a model-based SENSory imitatOR (SENSOR) to automatically change the agent's perspective to match the expert's. SENSOR jointly learns a world model to capture the dynamics of latent states, a sensor policy to control the camera, and a motor policy to control the agent. Experiments on visual locomotion tasks show that SENSOR can efficiently simulate the expert's perspective and strategy, and outperforms most baseline methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Apprenticeship learning via inverse reinforcement learning. In Proceedings of the twenty-first international conference on Machine learning, pp.  1, 2004.
  2. Bajcsy, R. Active perception. Proceedings of the IEEE, 76(8):966–1005, 1988.
  3. Model-based adversarial imitation learning. arXiv preprint arXiv:1612.02179, 2016.
  4. Mutual information neural estimation. In International conference on machine learning, pp. 531–540. PMLR, 2018.
  5. Analysis of representations for domain adaptation. Advances in neural information processing systems, 19, 2006.
  6. Domain-robust visual imitation learning with mutual information constraints. arXiv preprint arXiv:2103.05079, 2021.
  7. Active vision in robotic systems: A survey of recent developments. The International Journal of Robotics Research, 30(11):1343–1377, 2011.
  8. Reinforcement learning of active vision for manipulating objects under occlusions. In Conference on Robot Learning, pp.  422–431. PMLR, 2018.
  9. One-shot visual imitation learning via meta-learning. In Conference on robot learning, pp.  357–368. PMLR, 2017.
  10. Learn what matters: cross-domain imitation learning with task-relevant embeddings. Advances in Neural Information Processing Systems, 35:26283–26294, 2022.
  11. Learning robust rewards with adversarial inverse reinforcement learning. arXiv preprint arXiv:1710.11248, 2017.
  12. Distributed reinforcement learning of targeted grasping with active vision for mobile manipulators. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.  9712–9719. IEEE, 2020.
  13. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pp. 1180–1189. PMLR, 2015.
  14. Towards third-person visual imitation learning using generative adversarial networks. In 2022 IEEE International Conference on Development and Learning (ICDL), pp.  121–126. IEEE, 2022.
  15. A divergence minimization perspective on imitation learning methods. In Kaelbling, L. P., Kragic, D., and Sugiura, K. (eds.), 3rd Annual Conference on Robot Learning, CoRL 2019, Osaka, Japan, October 30 - November 1, 2019, Proceedings, volume 100 of Proceedings of Machine Learning Research, pp.  1259–1277. PMLR, 2019.
  16. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
  17. Learning to look by self-prediction. Trans. Mach. Learn. Res., 2023, 2023.
  18. Soft actor-critic algorithms and applications. CoRR, abs/1812.05905, 2018.
  19. Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv:1912.01603, 2019a.
  20. Learning latent dynamics for planning from pixels. In Chaudhuri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp.  2555–2565. PMLR, 2019b.
  21. Improving neural networks by preventing co-adaptation of feature detectors. CoRR, abs/1207.0580, 2012.
  22. Generative adversarial imitation learning. Advances in neural information processing systems, 29, 2016.
  23. Human-level performance in 3d multiplayer games with population-based reinforcement learning. Science, 364(6443):859–865, 2019.
  24. Domain adaptive imitation learning. In International Conference on Machine Learning, pp. 5286–5295. PMLR, 2020.
  25. Cross domain imitation learning. 2019.
  26. Active third-person imitation learning. arXiv preprint arXiv:2312.16365, 2023.
  27. Infogail: Interpretable imitation learning from visual demonstrations. Advances in neural information processing systems, 30, 2017.
  28. Visual learning by imitation with motor representations. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 35(3):438–449, 2005.
  29. Algorithms for inverse reinforcement learning. In Icml, volume 1, pp.  2, 2000.
  30. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  31. Visual adversarial imitation learning using variational models. Advances in Neural Information Processing Systems, 34:3016–3028, 2021.
  32. Cross-domain imitation from observations. In International Conference on Machine Learning, pp. 8902–8912. PMLR, 2021.
  33. Deep reinforcement learning framework for autonomous driving. arXiv preprint arXiv:1704.02532, 2017.
  34. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839):604–609, 2020.
  35. Self-supervised disentangled representation learning for third-person imitation learning. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.  214–221. IEEE, 2021.
  36. Active reinforcement learning under limited visual observability. arXiv preprint arXiv:2306.00975, 2023.
  37. Third-person visual imitation learning via decoupled hierarchical controller. Advances in Neural Information Processing Systems, 32, 2019.
  38. Third-person imitation learning. arXiv preprint arXiv:1703.01703, 2017.
  39. Adversarial imitation learning from incomplete demonstrations. arXiv preprint arXiv:1905.12310, 2019.
  40. Imitation learning from visual data with multiple intentions. In International Conference on Learning Representations, 2018.
  41. Deepmind control suite. arXiv preprint arXiv:1801.00690, 2018.
  42. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pp.  5026–5033. IEEE, 2012.
  43. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474, 2014.
  44. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
  45. Semail: Eliminating distractors in visual imitation via separated models. arXiv preprint arXiv:2306.10695, 2023.
  46. End-to-end learning of driving models from large-scale video datasets. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  2174–2182, 2017.
  47. One-shot imitation from observing humans via domain-adaptive meta-learning. arXiv preprint arXiv:1802.01557, 2018.
  48. Maximum entropy inverse reinforcement learning. In Aaai, volume 8, pp.  1433–1438. Chicago, IL, USA, 2008.
  49. Task-relevant adversarial imitation learning. In Conference on Robot Learning, pp.  247–263. PMLR, 2021.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Kaichen Huang (4 papers)
  2. Minghao Shao (16 papers)
  3. Shenghua Wan (6 papers)
  4. Hai-Hang Sun (2 papers)
  5. Shuai Feng (49 papers)
  6. Le Gan (12 papers)
  7. De-Chuan Zhan (90 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now