Papers
Topics
Authors
Recent
Search
2000 character limit reached

Bidirectional Human Interactive AI Framework for Social Robot Navigation

Published 5 Apr 2024 in cs.RO | (2404.04069v2)

Abstract: Trustworthiness is a crucial concept in the context of human-robot interaction. Cooperative robots must be transparent regarding their decision-making process, especially when operating in a human-oriented environment. This paper presents a comprehensive end-to-end framework aimed at fostering trustworthy bidirectional human-robot interaction in collaborative environments for the social navigation of mobile robots. In this framework, the robot communicates verbally while the human guides with gestures. Our method enables a mobile robot to predict the trajectory of people and adjust its route in a socially-aware manner. In case of conflict between human and robot decisions, detected through visual examination, the route is dynamically modified based on human preference while verbal communication is maintained. We present our pipeline, framework design, and preliminary experiments that form the foundation of our proposition.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. T. B. Sheridan, “Human–robot interaction: status and challenges,” Human factors, vol. 58, no. 4, pp. 525–532, 2016.
  2. W. Samek, G. Montavon, S. Lapuschkin, C. J. Anders, and K.-R. Müller, “Explaining deep neural networks and beyond: A review of methods and applications,” Proceedings of the IEEE, vol. 109, no. 3, pp. 247–278, 2021.
  3. M. Brandao, G. Canal, S. Krivić, and D. Magazzeni, “Towards providing explanations for robot motion planning,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 3927–3933.
  4. M. Fox, D. Long, and D. Magazzeni, “Explainable planning,” arXiv preprint arXiv:1709.10256, 2017.
  5. S. Almagor and M. Lahijanian, “Explainable multi agent path finding,” in AAMAS, 2020.
  6. J. Kottinger, S. Almagor, and M. Lahijanian, “Maps-x: Explainable multi-robot motion planning via segmentation,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 7994–8000.
  7. ——, “Conflict-based search for explainable multi-agent path finding,” in Proceedings of the International Conference on Automated Planning and Scheduling, vol. 32, 2022, pp. 692–700.
  8. D. Das, S. Banerjee, and S. Chernova, “Explainable ai for robot failures: Generating explanations that improve user assistance in fault recovery,” in Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, 2021, pp. 351–360.
  9. M. Diehl and K. Ramirez-Amaro, “A causal-based approach to explain, predict and prevent failures in robotic tasks,” Robotics and Autonomous Systems, vol. 162, p. 104376, 2023.
  10. G. Angelopoulos, A. Rossi, C. Di Napoli, and S. Rossi, “You are in my way: non-verbal social cues for legible robot navigation behaviors,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022, pp. 657–662.
  11. C. Mavrogiannis, A. M. Hutchinson, J. Macdonald, P. Alves-Oliveira, and R. A. Knepper, “Effects of distinct robot navigation strategies on human behavior in a crowded environment,” in 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI).   IEEE, 2019, pp. 421–430.
  12. P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Graph attention networks,” arXiv preprint arXiv:1710.10903, 2017.
  13. Y. Che, A. M. Okamura, and D. Sadigh, “Efficient and trustworthy social navigation via explicit and implicit robot–human communication,” IEEE Transactions on Robotics, vol. 36, no. 3, pp. 692–707, 2020.
  14. P. Kothari, S. Kreiss, and A. Alahi, “Human trajectory forecasting in crowds: A deep learning perspective,” IEEE Transactions on Intelligent Transportation Systems, pp. 1–15, 2021.
  15. K. Charalampous, I. Kostavelis, and A. Gasteratos, “Recent trends in social aware robot navigation: A survey,” Robotics and Autonomous Systems, vol. 93, pp. 85–104, 2017.
  16. A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese, “Social lstm: Human trajectory prediction in crowded spaces,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 961–971.
  17. J. Oh, J. Heo, J. Lee, G. Lee, M. Kang, J. Park, and S. Oh, “Scan: Socially-aware navigation using monte carlo tree search,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 7576–7582.
  18. D. Helbing and P. Molnar, “Social force model for pedestrian dynamics,” Physical review E, vol. 51, no. 5, p. 4282, 1995.
  19. C. W. Reynolds, “Flocks, herds and schools: A distributed behavioral model,” in Proceedings of the 14th annual conference on Computer graphics and interactive techniques, 1987, pp. 25–34.
  20. K. Mangalam, Y. An, H. Girase, and J. Malik, “From goals, waypoints & paths to long term human trajectory forecasting,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 15 233–15 242.
  21. Y.-J. Mun, M. Itkina, S. Liu, and K. Driggs-Campbell, “Occlusion-aware crowd navigation using people as sensors,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 12 031–12 037.
  22. V. Narayanan, B. M. Manoghar, R. P. RV, and A. Bera, “Ewarenet: Emotion-aware pedestrian intent prediction and adaptive spatial profile fusion for social robot navigation,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 7569–7575.
  23. Y. F. Chen, M. Liu, M. Everett, and J. P. How, “Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning,” in 2017 IEEE international conference on robotics and automation (ICRA).   IEEE, 2017, pp. 285–292.
  24. Y. F. Chen, M. Everett, M. Liu, and J. P. How, “Socially aware motion planning with deep reinforcement learning,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2017, pp. 1343–1350.
  25. Y. Yildirim and E. Ugur, “Learning social navigation from demonstrations with deep neural networks.”
  26. X. Mo, Z. Huang, Y. Xing, and C. Lv, “Multi-agent trajectory prediction with heterogeneous edge-enhanced graph attention network,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 9554–9567, 2022.
  27. J. Yang, X. Sun, R. G. Wang, and L. X. Xue, “Ptpgc: Pedestrian trajectory prediction by graph attention network with convlstm,” Robotics and Autonomous Systems, vol. 148, p. 103931, 2022.
  28. H. Tang, P. Wei, J. Li, and N. Zheng, “Evostgat: Evolving spatiotemporal graph attention networks for pedestrian trajectory prediction,” Neurocomputing, vol. 491, pp. 333–342, 2022.
  29. F. Da and Y. Zhang, “Path-aware graph attention for hd maps in motion prediction,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 6430–6436.
  30. Z. Liu, Y. Zhai, J. Li, G. Wang, Y. Miao, and H. Wang, “Graph relational reinforcement learning for mobile robot navigation in large-scale crowded environments,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 8, pp. 8776–8787, 2023.
  31. Y. Zhou and J. Garcke, “Learning crowd behaviors in navigation with attention-based spatial-temporal graphs,” arXiv preprint arXiv:2401.06226, 2024.
  32. E. Escudie and L. M. J. Saraydaryan, “Attention graph for multi-robot social navigation with deep reinforcement learning,” in International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2024.
  33. F. K. Došilović, M. Brčić, and N. Hlupić, “Explainable artificial intelligence: A survey,” in 2018 41st International convention on information and communication technology, electronics and microelectronics (MIPRO).   IEEE, 2018, pp. 0210–0215.
  34. A. Bauer, D. Wollherr, and M. Buss, “Human–robot collaboration: a survey,” International Journal of Humanoid Robotics, vol. 5, no. 01, pp. 47–66, 2008.
  35. M. Kirtay, E. Oztop, M. Asada, and V. V. Hafner, “Modeling robot trust based on emergent emotion in an interactive task,” in 2021 IEEE International Conference on Development and Learning (ICDL).   IEEE, 2021, pp. 1–8.
  36. M. Kirtay, E. Oztop, A. K. Kuhlen, M. Asada, and V. V. Hafner, “Trustworthiness assessment in multimodal human-robot interaction based on cognitive load,” in 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN).   IEEE, 2022, pp. 469–476.
  37. M. B. Luebbers, A. Tabrez, K. Ruvane, and B. Hayes, “Autonomous justification for enabling explainable decision support in human-robot teaming,” Proceedings of Robotics: Science and Systems. Daegu, Republic of Korea. https://doi. org/10.15607/RSS, 2023.
  38. P. T. Singamaneni, P. Bachiller-Burgos, L. J. Manso, A. Garrell, A. Sanfeliu, A. Spalanzani, and R. Alami, “A survey on socially aware robot navigation: Taxonomy and future challenges,” The International Journal of Robotics Research, p. 02783649241230562, 2024.
  39. D. Bolya, C. Zhou, F. Xiao, and Y. J. Lee, “Yolact: Real-time instance segmentation,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9157–9166.
  40. H. Sak, A. Senior, and F. Beaufays, “Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition,” arXiv preprint arXiv:1402.1128, 2014.
  41. C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Lee et al., “Mediapipe: A framework for building perception pipelines,” arXiv preprint arXiv:1906.08172, 2019.
  42. M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, A. Y. Ng et al., “Ros: an open-source robot operating system,” in ICRA workshop on open source software, vol. 3, no. 3.2.   Kobe, Japan, 2009, p. 5.
  43. P. Regulation, “Regulation (eu) 2016/679 of the european parliament and of the council,” Regulation (eu), vol. 679, p. 2016, 2016.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free