Gazing at Failure: Investigating Human Gaze in Response to Robot Failure in Collaborative Tasks (2502.16899v1)

Abstract: Robots are prone to making errors, which can negatively impact their credibility as teammates during collaborative tasks with human users. Detecting and recovering from these failures is crucial for maintaining effective level of trust from users. However, robots may fail without being aware of it. One way to detect such failures could be by analysing humans' non-verbal behaviours and reactions to failures. This study investigates how human gaze dynamics can signal a robot's failure and examines how different types of failures affect people's perception of robot. We conducted a user study with 27 participants collaborating with a robotic mobile manipulator to solve tangram puzzles. The robot was programmed to experience two types of failures -- executional and decisional -- occurring either at the beginning or end of the task, with or without acknowledgement of the failure. Our findings reveal that the type and timing of the robot's failure significantly affect participants' gaze behaviour and perception of the robot. Specifically, executional failures led to more gaze shifts and increased focus on the robot, while decisional failures resulted in lower entropy in gaze transitions among areas of interest, particularly when the failure occurred at the end of the task. These results highlight that gaze can serve as a reliable indicator of robot failures and their types, and could also be used to predict the appropriate recovery actions.

Analyzing Human Gaze as an Indicator of Robot Failure in Collaborative Tasks

The paper by Tabatabaei et al. investigates the role of human gaze behavior in detecting robot failures during collaborative tasks. This research is particularly pertinent as robots are increasingly integrated into various contexts, such as manufacturing and domestic environments, where they work alongside humans. The primary focus of the paper is to understand how human gaze dynamics, which are non-verbal cues, can signal robot failures and influence human perception of robotic teammates.

The researchers conducted an empirical paper involving 27 participants who interacted with a mobile manipulator robot to solve Tangram puzzles. The robot was designed to simulate two types of failures: executional failures (where the robot paused for an extended period) and decisional failures (where the robot incorrectly placed a puzzle piece). These failures occurred either at the beginning or end of the task, with some groups receiving verbal acknowledgment of the failure from the robot and others not. This paper design allowed for a nuanced exploration of how failure type, timing, and acknowledgment impact human gaze behavior.

Key findings demonstrated significant variations in human gaze in response to robot failures. When faced with executional failures, participants exhibited increased gaze shifts and maintained a higher focus on the robot compared to decisional failures, where gaze transitions among different areas of interest were more stable. Moreover, the timing of the failure influenced gaze distribution; early failures led to more randomness in gaze patterns, while late failures prompted more direct attention toward the robot or task-relevant areas.

These results highlight the potential for gaze as a reliable indicator to detect and respond to robot failures, suggesting that gaze monitoring systems could be integrated into human-robot interaction protocols to enhance failure detection and recovery strategies. Furthermore, gaze behavior analysis could also aid in improving robot design to foster more intuitive and resilient collaborations with human partners.

The paper also sheds light on the interplay between human perception and robot failures. Participants rated robots higher on perceived intelligence and performance trust when failures were acknowledged, suggesting the importance of robots communicating awareness of their errors.

The implications of these findings extend to both practical and theoretical domains. On a practical level, they underline the necessity for robots to possess mechanisms that detect failures through human gaze, enabling real-time adjustments that preserve trust and cooperation. Theoretically, this research contributes to understanding human-robot interaction, emphasizing the role of non-verbal communication in mitigating the impacts of robotic error.

In looking ahead, further investigations could explore the integration of eye-tracking technology in real-world robotic applications and the development of sophisticated algorithms to interpret gaze data. Moreover, expanding the range of failure types and exploring different robotic contexts could provide a broader framework for understanding and enhancing human-robot collaborative efficiency.

In summary, this paper delivers insightful contributions to the field of human-robot interaction by systematically examining how identifying and addressing robotic failures through human gaze dynamics can substantially benefit collaborative environments.