Mixed Reality Rehabilitation

• Mixed reality for rehabilitation is a technology that fuses virtual and physical elements to create interactive, ecologically valid motor and cognitive training paradigms.
• It leverages real-time sensing, advanced game engines, and multimodal feedback to personalize therapy, boost patient engagement, and enable remote care.
• Clinical studies show significant improvements in functional recovery and adherence, though challenges such as high hardware costs and integration complexities persist.

Mixed reality (MR) for rehabilitation refers to the use of technologies that integrate virtual and physical content—encompassing both virtual reality (VR) and augmented reality (AR), as well as their merger within extended reality (XR) frameworks—to deliver interactive, ecologically valid, and data-rich motor and cognitive training paradigms for neuro-musculoskeletal and cognitive rehabilitation. MR rehabilitation systems leverage real-time sensing, spatial mapping, game engines, and multimodal feedback (visual, auditory, haptic, psychophysiological) to enhance patient engagement, personalize therapy, and improve adherence, while enabling both in-clinic and remote, telemedicine-based care. This approach has applications across domains of motor recovery (post-stroke, prosthetics control, early mobilization, balance, and gait disorders), cognitive training, and geriatric and critical-care rehabilitation, with systems validated on both functional outcomes and user-centric metrics (Sun, 2022, Marozau et al., 25 Jul 2025, Kandel et al., 18 Sep 2025, Eom et al., 9 Feb 2026, González-Erena et al., 14 Jan 2025, Baron et al., 2023, Ines et al., 2010).

1. Definitions, Architectures, and System Components

Mixed reality in rehabilitation encompasses a spectrum:

• Virtual reality (VR): Fully immersive computer-generated environments with no direct physical world view; user interaction is typically mediated by head-mounted displays (HMDs) and hand controllers. VR is used for controlled scenario-based motor and cognitive training (Sun, 2022, Wang et al., 2019, Marozau et al., 25 Jul 2025).
• Augmented reality (AR): Overlays digital content onto the user's real-world view via see-through displays or passthrough video. AR supports direct visualization of patient anatomy, exercise trajectories, and contextual guidance over the user’s workspace (Sun, 2022, Marozau et al., 25 Jul 2025).
• Mixed reality (MR): Merges and anchors virtual objects within the physical environment to enable bidirectional interaction between real and digital elements, maintaining ecological validity and supporting functional task transfer (González-Erena et al., 14 Jan 2025, Kandel et al., 18 Sep 2025, Funke et al., 2024).

Systems are typically constructed from:

• Hardware: HMDs (HTC Vive, Oculus Rift/Quest/Pro, Meta Quest 3, Microsoft HoloLens 2), motion trackers (IMUs, optical trackers, Leap Motion, Wiimote), smart wearables (e.g., carbon nanotube sleeves), haptic devices.
• Software: Unity3D/Unreal Engine, XR SDKs (OpenXR, SteamVR), custom signal-processing and feedback modules, client-server or direct streaming architectures for data and telepresence (Sun, 2022, Eom et al., 9 Feb 2026, Baron et al., 2023, Ines et al., 2010).
• Sensing modalities: Motion (joint, hand, body tracking), EMG (for myoelectric control/training and engagement), EEG (cognitive workload adaptation), GSR (arousal metric), video and depth cameras (Sun, 2022, Marozau et al., 25 Jul 2025, González-Erena et al., 14 Jan 2025).

2. Therapeutic Workflows and Task Design

MR rehabilitation interventions are characterized by task paradigms that focus on restoration and adaptation of upper-limb, lower-limb, and postural function:

• Motor rehabilitation: Functional reaching, grasping, manipulation, and balance exercises are mapped onto ecological tasks emulated in VR/MR (e.g., PHAM object manipulation (Sun, 2022), Reach & Stack games (Marozau et al., 25 Jul 2025), balance training with floor-anchored waypoints (González-Erena et al., 14 Jan 2025), postural control in simulated urban scenes (Wang et al., 2019)).
• Prosthetic training: MR integrates real-time myoelectric control (sEMG via Myo armband) with visual and haptic feedback, supporting practice both with virtual and physical (bypass) prostheses (Sun, 2022).
• Cognitive-motor tasks: MR allows for dual-task walking, memory-object location drills, and cognitive load adaptation using multimodal psychophysiological feedback (EEG, GSR, ET) (González-Erena et al., 14 Jan 2025).
• Geriatric/Remote rehabilitation: MR telepresence and gamified light body-movement games directly address adherence barriers and memory challenges among older adults, with dynamic adjustment for fatigue and motivation (Kandel et al., 18 Sep 2025).

Task segmentation typically includes reach, manipulation/relocation, and return phases, instrumented with precise temporal, kinematic, and physiological monitoring (Sun, 2022, Eom et al., 9 Feb 2026).

3. Quantitative Metrics and Evaluation Protocols

Performance in MR rehabilitation is evaluated via objective kinematic, physiological, and subjective engagement metrics:

• Kinematic metrics: Task completion rate (CR), movement time (MT, segmented by phase), Fitts’ throughput (TP), path efficiency (PE), range of motion (ROM), mean joint velocities, workspace convex hull volume (Sun, 2022, Eom et al., 9 Feb 2026, González-Erena et al., 14 Jan 2025).
• Physiological/psychophysiological: EMG-based muscle activation (for engagement), EEG-derived cognitive workload indices (frontal theta/beta), GSR-derived arousal, heart rate and blood pressure (for exertion safety in ICU/critical care) (Marozau et al., 25 Jul 2025, Eom et al., 9 Feb 2026, González-Erena et al., 14 Jan 2025).
• Subjective/user metrics: System Usability Scale (SUS), Game Experience Questionnaire (GEQ), enjoyment/adherence ratings, visual-proprioceptive ownership scales (Ines et al., 2010, Kandel et al., 18 Sep 2025, Xiong et al., 2024).
• Clinical scales: Fugl-Meyer Assessment (FMA), Berg Balance Scale (BBS), Timed Up and Go (TUG), Activities-Specific Balance Confidence (ABC), Functional Gait Analysis (FGA), WOMAC index, Visual Vertigo Analog Scale, State and Trait Anxiety Inventory (Marozau et al., 25 Jul 2025, Wang et al., 2019, Sun, 2022).

Protocols include blocks of repeated tasks, randomized trial orders, baseline calibration (e.g., personalized motion boundaries), and both in-clinic and remote/user-home contexts (Sun, 2022, Baron et al., 2023, Kandel et al., 18 Sep 2025).

4. Feedback, Adaptation, and Multimodal Integration

MR systems employ dense, real-time feedback and adaptation to drive motor learning, engagement, and safety:

• Visual/auditory/haptic feedback: Color-coded cues, trajectory overlays, real-time annotation, virtual avatars with audio instructions, haptic actuators for proprioceptive input (vibrotactile or electrotactile bands), immersive 3D sound spatialization (Eom et al., 9 Feb 2026, Ines et al., 2010, González-Erena et al., 14 Jan 2025).
• Error-driven adaptation: Real-time joint-angle error detection, path deviation monitoring, adaptive guidance gain, and threshold-based corrective cues (Funke et al., 2024).
• Cognitive-physiological adaptation: Task difficulty (target size, temporal windows, stimulus density) modulated based on EEG/GSR/EMG/ET signals, maintaining patients in an optimal workload/arousal regime (González-Erena et al., 14 Jan 2025).
• Ownership and motivation: VR hand redirection (linear/post-offset transforms) invisibly assists otherwise unreachable tasks, maintaining high sense of embodiment and success-driven motivation (Xiong et al., 2024).
• Personalized calibration: Level and trajectory bounds mapped to baseline abilities; difficulty scaling for fatigue or motivational shifts (Eom et al., 9 Feb 2026, Kandel et al., 18 Sep 2025).

A derived, conceptual modular structure for MR rehabilitation systems is:

$$\text{MR\_PT\_System} = \{\,\text{Sensing: motion\_capture} \wedge \text{physiological\_sensors},\; \text{Modeling: 3D\_reconstruction}(\theta_{joints}, \rho_{segments}),\; \text{Feedback: visual\_annotations} \vee \text{haptic\_cues} \vee \text{gamified\_UI},\; \text{Adaptation: adjust}(\text{exercise\_parameters}, \text{user\_state})\,\}$$

(Kandel et al., 18 Sep 2025)

5. Clinical Outcomes, Benefits, and Limitations

Meta-analyses and controlled trials of MR in rehabilitation have demonstrated:

• Efficacy: Statistically significant gains in upper-limb and balance measures (FMA: Δ+6.3, BBS: Δ+4.1, TUG: Δ–2.2s, p<0.05–0.01), higher practice volume, and >85% session completion—a consistent improvement versus standard therapy, with medium-to-large effect sizes (Cohen’s d > 0.8) (Marozau et al., 25 Jul 2025, González-Erena et al., 14 Jan 2025, Sun, 2022).
• User engagement and adherence: Immersive, gamified and socially-rich MR environments drive higher enjoyment and motivation, supporting longitudinal therapy and home application; group MR telepresence increases social connectedness (Ines et al., 2010, Eom et al., 9 Feb 2026, Kandel et al., 18 Sep 2025).
• Safety and adaptability: Quantitative evaluation in critical care (ICU) confirms safe dosing (HR increase <10 BPM, SpO₂ drop <2%) and feasible deployment with high usability scores (Eom et al., 9 Feb 2026).
• Ecological validity: MR task anchoring in real-world context enhances transfer to functional daily activities and supports clinician/therapist monitoring at both macro (task success, gait symmetry) and micro (joint-angle, movement smoothness) levels (Sun, 2022, González-Erena et al., 14 Jan 2025, Kandel et al., 18 Sep 2025).

However, key challenges persist:

• Hardware cost and ergonomics: High cost (HoloLens 2 ~$3500), device weight (e.g., HoloLens 2 = 566g), and infrastructure requirements limit broad deployment (González-Erena et al., 14 Jan 2025, Marozau et al., 25 Jul 2025).
• Haptic feedback fidelity: Present solutions (vibrotactors/passive gloves) are limited compared to actual force cues; soft-robotic and wearable force-feedback remains an active area (Marozau et al., 25 Jul 2025, González-Erena et al., 14 Jan 2025).
• Technical limitations: Need for calibration, accurate joint tracking (≤5mm, 60Hz benchmark), and robust, low-latency networking (<50–100ms end-to-end) (Kandel et al., 18 Sep 2025).
• Multimodal integration and accessibility: Underutilization of multimodal (EEG/GSR/ET/haptic) feedback; accessibility barriers for older users and non-technical clinicians (González-Erena et al., 14 Jan 2025, Kandel et al., 18 Sep 2025).

6. Specialized Populations and Settings

MR rehabilitation is tailored across diverse populations and scenarios:

• Older adults: MR combats memory/fatigue/mobility challenges via visual feedback, telepresence, and adaptive gamification with design focus on comfort, simplicity, and minimized cognitive load (Kandel et al., 18 Sep 2025).
• Critical care (ICU): MR exergames titrate early mobilization in cardiovascularly unstable patients, integrating embodied avatars, variable motion boundaries, and real-time physiological monitoring (Eom et al., 9 Feb 2026).
• Prosthesis users: MR enables practice of myoelectric control in both virtual and AR-embedded contexts, with bypass shells encoding mass cues for realistic proprioceptive training (Sun, 2022).
• Telerehabilitation: Modular, low-cost MR (e.g., Wiimote tabletop projection, smart sleeve + VR) platforms promote at-home therapy and remote therapist oversight (Baron et al., 2023, Ines et al., 2010).

7. Future Directions and Open Challenges

Advancements in MR rehabilitation are expected in several key areas:

• Multimodal integration: Unified Bayesian frameworks for fusing EMG, EEG, GSR, and kinematics enable adaptive, personalized therapy progression (González-Erena et al., 14 Jan 2025).
• Low-cost, scalable platforms: Migration toward smartphone/tablet-based MR, embedded sensors in ordinary wearables, and cloud-based AI analytics seeks to democratize access (González-Erena et al., 14 Jan 2025, Marozau et al., 25 Jul 2025).
• Haptic and proprioceptive augmentation: Development of soft-robotic, high-fidelity wearable feedback for more complete embodiment (Marozau et al., 25 Jul 2025).
• Automated adaptation: Reinforcement learning and unsupervised calibration to minimize therapist workload and personalize to real-world conditions (Sun, 2022, Marozau et al., 25 Jul 2025).
• Protocol standardization: Need for consensus benchmarks, open datasets, and robust evaluation across diverse patient populations and clinical endpoints (Marozau et al., 25 Jul 2025, González-Erena et al., 14 Jan 2025).
• Privacy and security: Design for privacy-preserving real-time sensing and transmission, especially in remote/home applications for older or vulnerable populations (Kandel et al., 18 Sep 2025).

MR in rehabilitation thus represents a convergence of immersive technology, rich sensor fusion, and adaptive AI-driven feedback. It delivers quantifiable efficacy in functional recovery, while also addressing engagement, accessibility, and ecological validity across clinical and home settings (Sun, 2022, Marozau et al., 25 Jul 2025, González-Erena et al., 14 Jan 2025, Eom et al., 9 Feb 2026, Kandel et al., 18 Sep 2025).