2000 character limit reached
Inferring Belief States in Partially-Observable Human-Robot Teams (2403.11955v2)
Published 18 Mar 2024 in cs.RO and cs.HC
Abstract: We investigate the real-time estimation of human situation awareness using observations from a robot teammate with limited visibility. In human factors and human-autonomy teaming, it is recognized that individuals navigate their environments using an internal mental simulation, or mental model. The mental model informs cognitive processes including situation awareness, contextual reasoning, and task planning. In teaming domains, the mental model includes a team model of each teammate's beliefs and capabilities, enabling fluent teamwork without the need for explicit communication. However, little work has applied team models to human-robot teaming. In this work we compare the performance of two models, logical predicates and LLMs, at estimating user situation awareness over varying visibility conditions. Our results indicate that the methods are largely resilient to low-visibility conditions in our domain, however opportunities exist to improve their overall performance.
- Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
- Fuzzy mental model finite state machines: A mental modeling formalism for assessing mode confusion and human-machine “trust”. In 2022 IEEE 3rd International Conference on Human-Machine Systems (ICHMS), pages 1–4. IEEE, 2022.
- Shared mental models in expert team decision making. Current issues in individual and group decision making. Lawrence Erlbaum, Hillsdale, NJ, pages 221–246, 1993.
- On the utility of learning about humans for human-ai coordination. Advances in neural information processing systems, 32, 2019.
- Human-robot teaming: Perspective on analysis and implementation issues. In AIRO@ AI* IA, pages 12–17, 2018.
- State2explanation: Concept-based explanations to benefit agent learning and user understanding. Advances in Neural Information Processing Systems, 36:67156–67182, 2023.
- Improving human-robot team performance with proactivity and shared mental models. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, pages 2322–2324, 2023.
- Mica R Endsley. Design and evaluation for situation awareness enhancement. In Proceedings of the Human Factors Society annual meeting, volume 32, pages 97–101. Sage Publications Sage CA: Los Angeles, CA, 1988.
- Mica R Endsley. Situation awareness global assessment technique (sagat). In Proceedings of the IEEE 1988 national aerospace and electronics conference, pages 789–795. IEEE, 1988.
- Shared mental models to support distributed human-robot teaming in space. In 2018 aiaa space and astronautics forum and exposition, page 5340, 2018.
- Toward genuine robot teammates: Improving human-robot team performance using robot shared mental models. In Aamas, pages 429–437, 2020.
- The impact of multimodal communication on a shared mental model, trust, and commitment in human–intelligent virtual agent teams. Multimodal Technologies and Interaction, 2(3):48, 2018.
- Using simulations to model shared mental models. In Proceedings of the eighth international conference on cognitive modeling, pages 253–254. Taylor & Francis Psychology Press Ann Arbor, 2007.
- Team mental models: Techniques, methods, and analytic approaches. Human factors, 42(2):242–271, 2000.
- Human-robot teaming using shared mental models. ACM/IEEE HRI, 2012.
- Prolific. ac—a subject pool for online experiments. Journal of Behavioral and Experimental Finance, 17:22–27, 2018.
- The role of mental models in team performance in complex systems. IEEE transactions on systems, man, and cybernetics, 22(6):1296–1308, 1992.
- Matthias Scheutz. Computational mechanisms for mental models in human-robot interaction. In Virtual Augmented and Mixed Reality. Designing and Developing Augmented and Virtual Environments: 5th International Conference, VAMR 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA, July 21-26, 2013, Proceedings, Part I 5, pages 304–312. Springer, 2013.
- A framework for developing and using shared mental models in human-agent teams. Journal of Cognitive Engineering and Decision Making, 11(3):203–224, 2017.
- Mind meld: Personalized meta-learning for robot-centric imitation learning. In 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 157–165. IEEE, 2022.
- A research approach to shared mental models and situation assessment in future robot teams. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 55, pages 456–460. SAGE Publications Sage CA: Los Angeles, CA, 2011.
- A survey of mental modeling techniques in human–robot teaming. Current Robotics Reports, 1:259–267, 2020.
- An evaluation of human mental models of tactical robot movement. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 59, pages 1558–1562. SAGE Publications Sage CA: Los Angeles, CA, 2015.
- Tracking through containers and occluders in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13802–13812, 2023.