StretchBot: A Neuro-Symbolic Framework for Adaptive Guidance with Assistive Robots

Abstract: Assistive robots have growing potential to support physical wellbeing in home and healthcare settings, for example, by guiding users through stretching or rehabilitation routines. However, existing systems remain largely scripted, which limits their ability to adapt to user state, environmental context, and interaction dynamics. In this work, we present StretchBot, a hybrid neuro-symbolic robotic coach for adaptive assistive guidance. The system combines multimodal perception with knowledge-graph-grounded LLM reasoning to support context-aware adjustments during short stretching sessions while maintaining a structured routine. To complement the system description, we report an exploratory pilot comparison between scripted and adaptive guidance with three participants. The pilot findings suggest that the adaptive condition improved perceived adaptability and contextual relevance, while scripted guidance remained competitive in smoothness and predictability. These results provide preliminary evidence that structured actionable knowledge can help ground language-model-based adaptation in embodied assistive interaction, while also highlighting the need for larger, longitudinal studies to evaluate robustness, generalizability, and long-term user experience.

• The paper introduces a hybrid neuro-symbolic framework for adaptive assistive robotics by fusing LLM-based reasoning with real-time, multimodal perception.
• It employs a domain-specific knowledge graph and structured commands to effectively translate user context into precise robotic actions.
• Pilot evaluations show that adaptive routines enhance flexibility and object relevance, though they trade off smoothness and trust compared to scripted methods.

StretchBot: A Neuro-Symbolic Framework for Adaptive Guidance with Assistive Robots

System Overview and Neuro-Symbolic Architecture

StretchBot addresses the limitations of rigidly scripted assistive robotic systems by introducing a hybrid neuro-symbolic framework for adaptive user guidance during stretching routines. The architecture is partitioned into four stages: Perception, Reasoning, Action Execution, and Planning and Adaptation, establishing a real-time perception-reasoning-action loop.

Figure 1: The StretchBot pipeline is organized into four stages: Perception, Reasoning, Action Execution, and Planning and Adaptation. The arrows indicate the main information flow between blocks.

Multimodal Perception integrates object detection (YOLOv8n), affective signal analysis (voice, face, text-based), pose estimation (MediaPipe Pose), and ASR (Vosk STT) into a context package. The Reasoning module employs a domain-specific, JSON-based knowledge graph (KG) that encodes actionable relations among exercises, user states, object affordances, and routines, supplemented with ConceptNet for commonsense coverage. This curated KG grounds an LLM (deepseek-r1-0528-qwen3-8b), resulting in action commands utilizing structured prefixes (e.g., NEXT_EXERCISE, POINT_<OBJECT>, STOP_ROUTINE) and natural-language utterances.

The Action Execution layer translates symbolic instructions to robot arm control and synchronized TTS. Planning and Adaptation applies context-sensitive routine tracking, enabling session-level coherence while facilitating mid-routine adaptive branching (e.g., pausing, object suggestions) determined by dynamic user context.

Methodological Innovations

Distinct from prior LLM-for-robotics approaches relying on unstructured retrieval-augmented prompting, StretchBot fuses explicit symbolic context extracted from the KG and runtime perception into the LLM prompt. The information flow enables direct mapping from structured context to constrained, machine-actionable commands while maintaining human-eligible communication.

The emotion fusion pipeline applies late fusion of three modalities, yielding a single emotion label via reliability-weighted aggregation and $\operatorname{argmax}$ selection. Geometric pose verification employs per-exercise criteria without requiring supplementary data, ensuring instance-specific correctness and low infrastructural overhead.

Additionally, an optional verifier stage post-processes LLM outputs for command normalization and tone, trading minor delays for increased predictability and safety.

Exploratory Pilot Evaluation

The pilot evaluation (N=3) contrasts scripted versus adaptive routine executions, each comprising 8–10 minute sessions with crossover allocation to control for ordering effects. Key user experience dimensions (Clarity, Comfort, Adaptability, Trust, Naturalness, Object Relevance) were assessed via Likert questionnaires, complemented by post-session interviews.

Figure 2: Comparative results between scripted and adaptive conditions across the main user-evaluation dimensions.

Adaptive guidance obtained higher ratings for perceived adaptability (M=3.67 vs. 1.67) and object reference relevance (M=4.67 vs. 1.33), as expected from context-aware decision logic. Scripted routines outperformed in trust (M=3.33 vs. 2.67), comfort (M=4.67 vs. 4.33), and naturalness (M=3.33 vs. 3.00), highlighting user preference for smooth, low-cognitive-load interactions. Clarity was equivalent (M=4.00).

Qualitative interviews revealed a dichotomy: adaptive guidance was lauded for personalized responsiveness and attentiveness, but occasionally degraded flow and relaxational quality for users who preferred deterministic, low-branch routines. The implication is a user-dependent optimality in script-adaptive interpolation and the necessity of runtime mode adjustability.

Practical and Theoretical Implications

The positive object relevance and adaptability scores validate the use of explicit object affordance encoding and emotion/user-state grounding in robotic assistive routines. The decrement in trust and naturalness, however, underscores a critical trade-off when introducing LLM-based adaptation—namely, the cost in predictability, latency, and cognitive demand. The verifier module’s minimal but nonzero correction rate suggests feasibility of online monitoring for safety and format assurance, but the observed delay (3–10s) is a deployment-consequential obstacle.

StretchBot’s systematic knowledge graph curation demonstrates effective domain-restricted representation for typical stretches and user states, while integration with general-purpose KGs like ConceptNet covers the long tail of user/environmental cues. However, KG maintenance overhead and knowledge drift are ongoing concerns for scaling and real-world generalizability.

The findings further implicate that hybrid neuro-symbolic designs—rather than pure LLM or symbolic architectures—are well poised for embodied assistance tasks requiring both (i) grounded, constraint-safe control and (ii) flexible, affect-laden adaptation (2604.00628), [Ahn2022].

Limitations and Prospective Directions

Evaluation is restricted by small N, limited interaction diversity, and laboratory control of environmental variables. Scaling to longitudinal, in-the-wild deployments and diverse user populations will be required to validate robustness, longevity, and acceptance dynamics. User-state estimation could be expanded via integration of physiological sensors; routine adaptation policies might be co-learned from multi-session interactions.

On the technical side, latency could be mitigated by on-device LLMs or lighter neuro-symbolic ensembles. Automated KG population and self-adaptation to user preference policies represent canonical next steps. Additionally, future iterations should formally separate safety-critical from advisory logic and enhance user interface mechanisms for control and override in ambiguous scenarios.

Conclusion

StretchBot offers tangible evidence that neuro-symbolic architectures—explicit KG representations grounded in multimodal real-time perception and governed by LLM-based contextual reasoning—can support context-aware, adaptive assistive robots with domain-constrained flexibility. The core trade-off between personalization and routine smoothness emerges as user-dependent: optimal deployment may necessitate adjustable adaptation-scripted blending, safety-focused design, and interpretable, auditable logic.

As the domain matures, advances in hybrid knowledge representation, adaptive user modeling, and robust real-time reasoning stand as the critical axes for furthering grounded, user-centered assistive robotics.