Where Do We Look When We Teach? Analyzing Human Gaze Behavior Across Demonstration Devices in Robot Imitation Learning

Abstract: Imitation learning for acquiring generalizable policies often requires a large volume of demonstration data, making the process significantly costly. One promising strategy to address this challenge is to leverage the cognitive and decision-making skills of human demonstrators with strong generalization capability, particularly by extracting task-relevant cues from their gaze behavior. However, imitation learning typically involves humans collecting data using demonstration devices that emulate a robot's embodiment and visual condition. This raises the question of how such devices influence gaze behavior. We propose an experimental framework that systematically analyzes demonstrators' gaze behavior across a spectrum of demonstration devices. Our experimental results indicate that devices emulating (1) a robot's embodiment or (2) visual condition impair demonstrators' capability to extract task-relevant cues via gaze behavior, with the extent of impairment depending on the degree of emulation. Additionally, gaze data collected using devices that capture natural human behavior improves the policy's task success rate from 18.8% to 68.8% under environmental shifts.

• The paper shows that wearable devices with natural human embodiment significantly improve gaze focus, increasing policy success from 18.8% to 68.8%.
• The paper identifies that robotic emulation devices induce higher cognitive and physical workloads, as reflected in elevated NASA-TLX scores.
• The paper reveals that egocentric views combined with gaze-focused instructions enable better task cue extraction compared to top-down perspectives.

Analyzing Human Gaze Behavior in Robot Imitation Learning Across Demonstration Devices

In the study titled "Where Do We Look When We Teach? Analyzing Human Gaze Behavior Across Demonstration Devices in Robot Imitation Learning," researchers Ishida et al. explore the influence of various demonstration devices on human gaze behavior within the context of imitation learning for robotics. Imitation learning is a process by which robots acquire manipulation skills through human demonstration, typically necessitating extensive demonstration data to achieve generalizable policies. However, collecting such data can be costly and labor-intensive. The paper hypothesizes that leveraging human cognitive abilities, particularly eye-gaze patterns, could enhance the capture of task-relevant cues, thereby improving the efficacy of imitation learning from relatively small datasets.

The research specifically examines how demonstration devices, which may emulate a robot's embodiment or visual conditions, affect the demonstrators' gaze behavior and subsequent task cue extraction. The authors propose an experimental framework to systematically assess gaze behaviors with a range of devices, from those that preserve natural human capabilities to those that mimic robotic constraints. The study identifies two primary factors influencing gaze behavior: the emulation of a robot's physical embodiment and the visual perspective provided to the demonstrator.

Key Findings

1. Embodiment and Gaze Efficiency: The research reveals that demonstration devices with a more natural human embodiment, such as wearable cameras, enhance the demonstrators' ability to focus on task-relevant cues more effectively than devices that impose robotic constraints. For instance, the study found that the success rate of policies trained with gaze data from devices mimicking natural behavior improved from 18.8% to 68.8% under varying environmental conditions.
2. Workload Comparisons: Devices emulating robotic embodiment, such as the Leader and Leader-Follower systems, introduced higher cognitive and physical workloads compared to wearable devices. These findings align with the increased workload scores observed on NASA-TLX sub-indices, suggesting the importance of maintaining natural human capabilities in task demonstrations to mitigate participant burden.
3. Visual Conditions and Instructional Support: The visual perspective provided by devices, such as head-mounted displays offering top-down views, adversely impacted task-relevant cue extraction when compared to egocentric views. Interestingly, providing gaze-relevant instructions ameliorated some of these challenges, particularly aiding in aligning gaze with task objectives.

Implications and Future Directions

This research underscores the importance of selecting appropriate demonstration devices in imitation learning frameworks to optimize gaze-based cue extraction. It suggests a dual strategy: utilize wearable cameras for natural gaze data collection and leader-follower systems for gathering consistent demonstration data. This approach balances the trade-offs between embodiment fidelity and the reduction of domain gaps relevant to policy training.

From a practical standpoint, the study highlights potential pathways to reduce data requirements for training robotic systems, thereby lowering costs and accelerating developmental cycles. Theoretically, it opens new avenues for integrating cognitive science insights, such as gaze patterns and task cue prioritization, into the design of robotic learning protocols.

Future directions may include developing advanced policy architectures capable of simultaneously processing gaze behavior in conjunction with action sequences, as well as exploring gaze-based subgoal identification for hierarchical learning models. Additionally, extending this analysis to more complex and dexterous tasks will further validate the efficacy of gaze-informed imitation learning approaches.

Overall, this paper contributes significant insights into the intersection of human cognitive behaviors and robotic learning, offering valuable guidance for enhancing the robustness and efficiency of imitation learning strategies through human gaze analysis.