CANINE: Coaching Visually Impaired Users for Interactive Navigation with a Robot Guide Dog

Abstract: Robot guide dogs offer navigation assistance that greatly expands the independent mobility of the visually impaired, but their effective use requires subtle human-robot coordination that is difficult for users to learn from generic verbal instructions. To tackle this challenge, we present CANINE, an automated coaching system that trains users for interactive navigation with a robot guide dog, through personalized, adaptive verbal feedback. CANINE decomposes a complex coordination task into sub-skills and operates at two levels. At the high level, it decides what to train by tracking the learner's proficiency across sub-skills using knowledge tracing and prioritizing training on the weakest areas. At the low level, CANINE decides how to train each sub-skill by observing each human practice episode, using foundation models to infer the underlying causes of errors, and generating targeted verbal corrections adaptively. A controlled study with blindfolded participants, treated as a proxy population for quantitative evaluation, demonstrates that CANINE significantly improves both learning efficiency and final navigation performance compared to generic verbal instructions. We further validate CANINE through a retention study and an exploratory case study. The retention study shows lasting skill improvement after two weeks. The case study confirms CANINE's effectiveness in training a visually impaired user, while revealing additional design considerations for real-world deployment. Both are well aligned with the findings of the controlled study. Project page: https://cunjunyu.github.io/project/canine/

• The paper presents a hierarchical, adaptive coaching system that decomposes robot-assisted navigation into distinct sub-skills for visually impaired users.
• It employs Bayesian knowledge tracing and vision-language models to dynamically generate personalized verbal feedback from first-person camera inputs.
• Experimental evaluations show significant improvements in task completion times and persistent skill retention compared to traditional instruction methods.

CANINE: Automated Coaching for Visually Impaired Navigation with Robot Guide Dogs

Motivation and Problem Statement

The proliferation of assistive robotic platforms offers significant promise for improving independent mobility for visually impaired (VI) individuals. However, the effective use of these platforms, particularly robot guide dogs, demands intricate human-robot coordination that cannot be acquired through generic instructions alone. The central challenge is twofold: task decomposition is required because navigation tasks consist of heterogeneous sub-skills, and VI users lack visual feedback, which complicates error diagnosis and self-correction. CANINE addresses this by providing automated, adaptive coaching with personalized verbal feedback, targeting the development of coordination skills crucial for safe and efficient robot-assisted navigation.

Figure 1: CANINE. In our study, a robot guide dog coaches a visually impaired user to navigate through a doorway.

Conceptual Framework and System Architecture

CANINE adopts a two-level hierarchical coaching architecture. The system decomposes navigation tasks, such as doorway traversal, into modular sub-skills: "Navigate to Door," "Open Door," and "Enter Room." The upper layer (inter-skill coaching) utilizes Bayesian knowledge tracing to estimate latent proficiency across sub-skills and dynamically sequences training based on weakest components. The lower layer (intra-skill coaching) processes first-person video streams, infers behavioral and error states using Vision-LLMs (VLMs), and generates actionable instructions via LLMs. Symbolic representations and episode summaries are leveraged to ensure explicit reasoning and minimize hallucinations, with outputs constrained by predefined JSON schemas.

Figure 2: Expert coach insights from formative study. Analysis reveals common learner challenges, effective coaching strategies (timing and content), and design implications for automated coaching systems.

Figure 3: Overview of CANINE. CANINE employs a two-level coaching strategy. The inter-skill coaching (up-right) tracks proficiency across sub-skills using knowledge tracing and selects the sub-skill to practice next. The intra-skill coaching (down-right) takes in video observations and generates coaching instructions for the selected sub-skill.

Technical Implementation and Reasoning Pipeline

CANINE operates episodically, capturing user behavior through a chest-mounted camera. The intra-skill pipeline consists of four stages: frame-level state extraction (using VLMs every 0.5 s), episode timeline summarization, coaching generation (LLM-based error diagnosis and instruction synthesis), and robot-side adaptation (parameter tuning for task difficulty). Feedback is delivered post-episode to avoid disruptions during safety-critical navigation.

Figure 4: Illustration of the summarized timeline. The timeline aggregates per-frame observations sampled every 0.5\,s into a structured episode-level summary.

Figure 5: Robot guide dog hardware setup. Unitree Go2 quadruped robot equipped with Jetson AGX Orin for onboard computation, Hesai JT16 LiDAR for navigation, and chest-mounted camera on the user for first-person view analysis.

Experimental Evaluation

Adaptive Curriculum and Component Feasibility

Simulated learner trials show that the adaptive curriculum achieves faster mastery of skill hierarchies compared to uniform or greedy selection. Bayesian proficiency tracking enables precise targeting of weakest sub-skills, minimizing teaching actions as sub-skill count increases.

Figure 6: Adaptive curriculum sequencing achieves faster learning. Violin plots show the distribution of teaching actions needed to reach mastery for different numbers of sub-skills. Adaptive curriculum sequencing achieves faster learning.

VLM Frame Analysis and Feedback Generation

Human judges compared frame analysis accuracy across multiple VLM architectures; GPT-5.1 demonstrated superior performance, with preference rates exceeding human annotation for task-critical frames. CANINE's staged pipeline delivered higher helpfulness, factuality, and specificity scores compared to baselines, with an average feedback latency of 20.62 s, suitable for terminal feedback regimes.

Figure 7: Frame analysis accuracy comparison. Preference rates of VLM-generated descriptions versus human annotations across different LLMs (GPT-5.1, Claude-4.5-Sonnet, Gemini-3.0-Pro amd Qwen3-VL).

Human Subject Results

In a controlled user study (N=20, blindfolded proxies), CANINE yielded significantly greater improvement in time-to-completion for interaction-intensive sub-skills ("Open Door": Δ = 8.64 s vs. 4.53 s, p = .012; "Enter Room": Δ = 6.96 s vs. 2.27 s, p = .019), and faster final completion times (all comparisons p < .05). The subjective evaluation showed CANINE is rated significantly higher in Perceived Usefulness (PU: M=13.10 vs. M=10.80, p=0.036) than baseline instruction.

Figure 8: Navigation task setup and experimental environment. We show the three sub-skills of door navigation: Navigate to Door, Open Door, and Enter Room. Top: human behavior. Bottom: visualization of the navigation map.

Figure 9: Subjective evaluation results. Participants rated CANINE as having significantly higher Perceived Usefulness (PU) compared to general instruction.

Figure 10: Qualitative examples of generated instructions. Top-left shows the image captured by the chest-mount camera.

Figure 11: User ratings of CANINE components. Using a 1 - 7 Likert scale (1 = not helpful, 7 = very helpful).

Retention Analysis and Case Study with Visually Impaired User

Skill retention persisted for at least two weeks post-training; participants maintained or improved proficiency in the most challenging sub-skill ("Open Door"), indicating consolidation effects. An exploratory case study with a VI user mirrored the proxy study results but highlighted additional requirements: multimodal haptic cues and feedback control features for autonomy (pause, replay, skip).

Figure 12: Skill retention. Bars show the mean retention change for each skill in second.

Extensibility to robot handover tasks was demonstrated, requiring concurrent feedback for fine-motor hand alignment, suggesting task-dependent adaptation of feedback timing strategies.

Figure 13: Experimental setup for the handover task. Human practices receiving objects from a robot manipulator.

Implications, Limitations, and Future Directions

CANINE establishes a robust pipeline for automated, adaptive coaching of VI users in embodied navigation tasks with robot guide dogs. The integration of knowledge tracing for curriculum sequencing and foundation models for error diagnosis/feedback generation enables task decomposition, targeted instruction, and explicit reasoning. Theoretical implications include the potential for scalable, personalized motor skill coaching in assistive robotics; practical deployments will require enhancements: comprehensive multimodal sensing, automated skill discovery, and longitudinal validation in VI populations.

The present approach is constrained by terminal feedback timing, reliance on chest-mounted camera (with occlusion and viewpoint issues), and expert-defined sub-skill decomposition. Further research should investigate concurrent feedback in dynamic tasks, sensor fusion, ablations of system components, and curriculum baselines with human-selected scheduling. Extension to complex navigation scenarios (crowds, elevators, public infrastructure) and fully autonomous task resetting will advance real-world applicability.

Conclusion

CANINE represents an integrated system for automated, hierarchical coaching in assistive robot navigation, demonstrating quantifiable performance gains and persistent skill retention. The architecture leverages foundation models and knowledge tracing for explicit proficiency modeling and adaptive feedback, setting the stage for scalable, personalized training for VI users. Continued investigation into sensor modalities, adaptive feedback timing, and longitudinal deployment will define the path forward for practical adoption and generalization to broader human-robot collaborative tasks.