Attune: A Self-Annotation Tool for Understanding Robot Operator Attention Profiles

Abstract: Deploying robot fleets in complex, real-world environments requires human operators to supervise multiple robots simultaneously. Managing operator attention is a fundamental challenge of designing multi-robot supervision interfaces, encompassing both feed layout and feed content (i.e., robot behavior design). Thus far, designers lack empirical guidance on the latter-how to change a robot's behavior to capture, sustain, or relinquish operator attention during multi-robot supervision. In our vision of the future, designers should be able to use this guidance to calibrate robot behavior to different operator attention profiles. Treating operator eye gaze as a robot behavior design clue, we created a pre-deployment elicitation tool called Attune. Attune automatically identifies when meaningful gaze shifts occur, provides AI assistance for annotating why shifts occurred, and outputs a summary of operator gaze patterns for operator review. We evaluated Attune through a user study in which participants annotated the visual triggers that drew their attention. Our findings unveil variation in observed gaze patterns and reveal how Attune helps characterize operator attention.