NeuroAR: AR in Neuroscience and Clinical Guidance

Updated 7 July 2026

NeuroAR is a multidisciplinary domain merging AR with neuroscience to deliver spatially registered overlays for clinical, educational, and imaging applications.
Innovations include markerless tracking, digital twin integration, and advanced registration methods that enhance precision in neurosurgery and brain stimulation.
Ongoing challenges involve ensuring tracking stability, feedback fidelity, and rigorous accuracy assessment to meet clinical and cognitive demands.

Searching arXiv for papers on NeuroAR and closely related usage. arxiv_search(query="NeuroAR augmented reality neurosurgery neuronavigation brain stimulation", max_results=10, sort_by="submittedDate") NeuroAR denotes a heterogeneous but increasingly coherent body of work at the intersection of augmented reality and neuroscience, neurosurgery, neuronavigation, brain stimulation, neuroeducation, and brain-computer interaction. In its predominant contemporary usage, NeuroAR refers to systems that spatially register anatomy, targets, instruments, or neurofeedback directly into the physical scene so that clinicians, trainees, or users do not rely exclusively on external monitors or abstract coordinate displays. The label is not fully uniform, however: alongside this AR-centered usage, “NeuroAR” also appears as the specific name of a generative autoregressive model for longitudinal brain MRI aging synthesis, and adjacent neuroimaging work has explicitly connected sparse-plus-low-rank autoregressive modeling to NeuroAR-style analysis of spatio-temporal structure (Bagheri et al., 2023, Ho et al., 2024, Yesiloglu et al., 29 Jul 2025, Liégeois et al., 2015).

1. Terminological scope and conceptual boundaries

In the AR-centered literature, NeuroAR is best understood as a domain label for neurotechnology in which AR functions as an operational interface rather than a purely illustrative overlay. Recent instances include AR-assisted neurosurgery assessment, markerless browser-based neurointerventional planning, tool-guided HoloLens navigation for external ventricular drain placement, smartphone- or headset-based TMS neuronavigation, hyperspectral AR guidance during tumor resection, ADHD-oriented AR cognitive support, and EEG-driven AR-robot interaction (Bagheri et al., 2023, Ho et al., 2024, Fischer et al., 14 Aug 2025, Hu et al., 28 Jan 2026, Sancho et al., 2024, Ghasemi et al., 2024, Wang et al., 25 Sep 2025).

This broad usage should be distinguished from title-specific acronymic uses. “NeuroAR” in “Neural Autoregressive Modeling of Brain Aging” is a 3D generative transformer for predicting future MRI scans from earlier scans, not an AR visualization system (Yesiloglu et al., 29 Jul 2025). A further adjacent usage appears in multivariate autoregressive neuroimaging time-series modeling, where sparse-plus-low-rank AR identification is explicitly connected to NeuroAR through its ability to recover dynamic latent components and spatio-temporal network structure from brain data (Liégeois et al., 2015). This suggests that the term has evolved around a shared concern with anatomically or functionally structured brain representations, even when the underlying modality differs.

A common misconception is therefore that NeuroAR refers only to head-mounted surgical overlays. The literature instead spans intraoperative guidance, planning, stimulation, education, therapy-like interaction design, and neural-interface feedback loops, with a secondary but distinct strand in autoregressive neuroimaging models (Ho et al., 2024, Jahani et al., 31 Mar 2026, Yesiloglu et al., 29 Jul 2025).

2. Clinical and technical rationale

The core clinical rationale for NeuroAR is the reduction of split attention between the patient or task space and a separate navigation display. In neurosurgery, conventional neuronavigation often requires the operator to alternate between the operative field and a remote monitor; in TMS, conventional systems frequently guide placement through external screens, abstract numbers, or 6D crosshairs. AR is introduced as a way to place anatomy, stimulation targets, or trajectories directly in situ, thereby reducing cognitive load and improving spatial understanding, but only if the overlay remains stable, registered, and trustworthy (Bagheri et al., 2023, Hu et al., 23 Jan 2026).

Across applications, several design principles recur. First is the use of a registered anatomical frame: head pose, phantom pose, or patient pose must be estimated and maintained so that virtual content remains aligned during motion. Second is the use of a “digital twin” in Unity or a related engine, with streamed tracking data driving a real-time 3D brain or tool model. Third is a preference in several papers for lower-cost or lower-complexity hardware stacks, such as consumer RGB cameras, AprilTags, smartphones, browser-based webcam pipelines, or HoloLens onboard sensors, in place of expensive infrared stereotaxy or motion-capture systems (Hu et al., 28 Jan 2026, Ho et al., 2024, Fischer et al., 14 Aug 2025).

The principal technical liabilities are also consistent across the literature. Neurosurgical AR is challenged by vergence-accommodation conflict, occlusion, hologram instability in space, lack of haptic feedback, and registration or tracking drift; TMS systems remain sensitive to calibration quality, marker visibility, lighting variation, and rapid motion; cognitive-support applications for ADHD explicitly warn that AR can worsen distraction and sensory overload if salience, pacing, and modality are not carefully managed (Bagheri et al., 2023, Hu et al., 28 Jan 2026, Ghasemi et al., 2024). The field is thus defined not by overlay alone, but by the attempt to make spatially registered, task-relevant augmentation operational under neuroclinically meaningful error tolerances.

3. Accuracy assessment and measurement frameworks

A central NeuroAR problem is not simply how to register a hologram, but how to assess the actual end-to-end accuracy of AR guidance under realistic use. In AR-assisted neurosurgery, prior assessment methods are classified into three families: external-device methods, post-operative CT methods, and user-perspective methods (Bagheri et al., 2023).

Assessment family	Typical mechanism	Principal bias or limitation
External device	Stylus or landmarks compared against tracked ground truth	External tracker must itself be validated
Post-operative CT	Drill hole or catheter tip measured in imaging	Direct but invasive or single-use
User perspective	Marking on paper/model or visual displacement estimation	Vulnerable to hologram instability and VAC

The most systematic methodology in this set is an external-device-based assessment pipeline built around calibrated Vicon cameras, a Unity3D application, and a Python server connected via TCP. The probe and phantom each carry retroreflective markers; CT-space and lab-space are linked through point-to-point rigid transformations computed from corresponding marker centers. The probe tip is reconstructed from CT with 1 mm axial cut accuracy and estimated via a fitted sphere model, while the phantom uses 9 retroreflective markers distributed across the head. The logic is explicitly decomposed into no interaction, interaction with holograms for planning, and operative interaction with physical tools under AR guidance, so that overall user error can be interpreted as a sum of separately measurable components rather than a single black-box metric (Bagheri et al., 2023).

That decomposition exposes uncertainty in the measurement apparatus itself. In the Vicon-based study, stationary sphere detection over more than 1400 frames showed standard deviation not exceeding 0.16 mm, sphere-to-sphere distance variation during probe motion had a maximum variance of 0.25 mm, and CT-based sphere fitting yielded a maximum standard deviation of 0.8 mm. The reported user-facing metrics then separate probe tip estimation error, ground-truth transformation error, and the final mean Euclidean distance between the user-indicated point and the ground-truth fiducial (Bagheri et al., 2023).

Later systems preserve the same end-to-end logic while changing the tracking substrate. HoloLens 2 navigation for simulated EVD placement begins with six anatomical landmarks and refines registration through surface tracing, KD-tree nearest-neighbor matching, inlier filtering, normal-based projection filtering, and multiscale ICP, rejecting alignments with Fitness below 0.7 or RMSE above 5 mm (Fischer et al., 14 Aug 2025). Low-cost TMS systems instead formulate pose estimation as a least-squares reprojection problem over AprilTag corners, then fuse multi-camera estimates through Gaussian precision-weighted averaging, making 2D reprojection error and propagated distance uncertainty part of the measurement chain rather than hidden implementation details (Hu et al., 28 Jan 2026).

Representative NeuroAR systems differ substantially in platform and sensing, but they converge on real-time pose estimation, anatomical registration, and live spatial feedback.

The markerless web-based neurointerventional planning system of Al Hamilton and colleagues is implemented as a browser pipeline combining MediaPipe for facial localization and segmentation, Three.js and React Three Fiber for rendering, and Leva for runtime parameter control. Its central task is to overlay a virtual head anatomy model and the 2-RPS parallel positioner, or Skull-Bot, on the user’s face without markers, dedicated AR headsets, or specialized local software. Auto-scaling matches the model projection boundary $B_m$ to the segmented head boundary $B_i$ through scale factors $s_w = \frac{w_i}{w_m}$ and $s_h = \frac{h_i}{h_m}$ , while real-time head tracking uses the MediaPipe face pose matrix $P_i$ . In a pilot study with three participants rotating the head in pitch, yaw, and roll by approximately 45 degrees, the reported mean width error was $10.09 \pm 5.53\%$ , mean height error was $7.60 \pm 5.53\%$ , and mean IoU was $80.87 \pm 4.42\%$ . The same paper notes that the method depends on visible facial features and is less suitable for cases such as hind-brain surgery training (Ho et al., 2024).

HoloLens 2 neurosurgical navigation with surface tracing pursues a different design. Registration begins with six landmarks—left and right intertragal notch, lateral canthi, and medial canthi—followed by stylus-based surface tracing under HoloLens 2 Research Mode. The guidance stage then separates static in-situ visualization from real-time tool-tracking guidance. Static mode renders a 6 mm disc at the entry point, a 1 mm cylinder for the planned trajectory, and a 4 mm sphere at the target. Tool-tracking mode adds continuous catheter tip localization, trajectory deviation indicators, corrective arrows, and projected depth-to-target feedback. On 20 3D-printed head phantoms filled with ballistic gel and tested by 9 intended users, tool tracking reduced angular deviation from 4.3° to 2.3°, target-tip error from 17.5 mm to 4.3 mm, target depth error from 14.5 mm to 2.2 mm, and target radial error from 6.9 mm to 2.5 mm; the overall SUS score was $78.5 \pm 13.2$ (Fischer et al., 14 Aug 2025).

In non-invasive brain stimulation, the AprilTag-based multi-camera TMS framework replaces conventional infrared neuronavigation with three CANYON CNE-CWC5 RGB cameras, four head markers, one coil marker, a Unity digital twin, and AR Foundation running on an Android smartphone such as the Xiaomi Mi 9. The total tracking hardware cost is reported as about £60. Across five positions with repeated measurements, distance precision was 0.07–0.09 mm, angular precision 0.04–0.06°, and absolute errors were below 0.5 mm for distance and 0.3° for rotation; average stimulation-point localization error across 15 test points was 4.94 mm, with 33.3% of measurements below 4 mm. In a case study with 10 medically trained participants, all completed alignment tasks successfully and the AR guidance was rated as easy to understand and clear to follow (Hu et al., 28 Jan 2026).

SLIMBRAIN extends NeuroAR beyond navigation geometry into intraoperative tissue characterization. It combines a hyperspectral snapshot camera, Intel RealSense L515 LiDAR, RGB imaging, GPU processing, and OpenGL point-cloud rendering so that tumor, healthy tissue, blood, and dura mater can be classified and overlaid in an AR point cloud during resection. The system captures and processes hyperspectral data at 14 FPS, verified in real brain tumor resection operations, and reports global AUC of 95.27% and tumor-class AUC of 95.17% (Sancho et al., 2024). Here AR is functioning as the mechanism that makes hyperspectral classification spatially interpretable during surgery.

5. Perception, feedback, and human factors

A major finding across NeuroAR is that static visual overlay is often not the decisive factor in performance; physical constraint, tracked tool feedback, and feedback congruence frequently dominate.

The clearest neurosurgical demonstration comes from the three-condition AR-assisted neurosurgery assessment study. In the no-feedback condition, mean error was 12.75 mm with SD 2.94 mm; in the holographic-feedback condition, mean error was 11.99 mm with SD 2.99 mm; and in the physical-feedback condition, mean error dropped to 6.98 mm with SD 3.04 mm. A Z-test comparing physical and holographic conditions yielded about 8.21 with $p \le 0.0001$ , whereas the comparison between holographic and no feedback gave $B_i$ 0 with $B_i$ 1. The reported interpretation is that physical contact substantially reduces the apparent effect of hologram displacement, while added visual holographic proximity feedback does not significantly improve users’ depth perception relative to seeing the hologram alone (Bagheri et al., 2023).

The EVD-navigation study reaches a closely related conclusion from a different experimental design. Real-time tool-tracking guidance outperformed static in-situ visualization across nearly all accuracy measures and was preferred in subjective evaluation, particularly for visualizing angulation, depth, and confidence in hitting the target. This directly challenges the assumption that overlaying a planned trajectory on anatomy is sufficient for neurosurgical execution; continuous relational feedback about the physical tool appears more effective than a fixed hologram when precision is paramount (Fischer et al., 14 Aug 2025).

In BCI-robot interaction, AR feedback operates not as surgical guidance but as a neurofeedback channel. The EEG-driven closed-loop AR-robot system uses a smartphone-based Unity interface in which the AR block and arrow sway slightly in the same direction as left or right motor imagery, while “lift” locks onto the selected target. Under this Neurofeedback condition, the reported accuracy was 96.9%, false positive rate 2.8%, and ITR 21.3 bits/min, outperforming No AR, Static, and Sham conditions; closed-loop grasping achieved a 97.2% success rate (Wang et al., 25 Sep 2025). This indicates that AR can stabilize control in a noisy neural decoding loop rather than merely visualize the robot state.

Human-factors work for ADHD frames the same issue more cautiously. AR is presented as a cognitive scaffolding medium that can guide attention, reduce irrelevant sensory load, and support working memory, but it can also worsen distraction and sensory overload if salience and modality are poorly tuned. The proposed design framework centers on human information processing and the SEEV model—Salience, Effort, Expectancy, and Value—and the prototype evaluation plan uses Apple Vision Pro eye tracking with SVC using an RBF kernel, best-reported at $B_i$ 2 and $B_i$ 3 (Ghasemi et al., 2024). AR-Therapist pushes this into a therapy-like architecture for ADHD, but its evidence remains simulation-based rather than clinical: the reported Performance Index rises from 44% to 72% as correct trials increase and errors decrease, and from 24% to 82% as engagement increases with no errors (Alqithami et al., 2020).

6. Neuroeducation and pedagogical NeuroAR

Education-oriented NeuroAR systems shift emphasis from procedural accuracy to spatial comprehension, curriculum design, and user engagement. NeuroVase is a tablet-based mobile AR platform for neurovascular anatomy and stroke education built around a dual-mode cue-card design: the same cards serve as standalone study aids and as Vuforia image targets that trigger 3D anatomical content. Its curriculum is organized into three modules—brain lobes, cerebral arterial system, and vascular territories—and the vascular-territory content is anchored to a Digital 3D brain MRI arterial territories atlas built from lesion distributions in 1,298 acute stroke patients. In a controlled study with 40 participants, both AR and paper-based groups improved significantly from pre- to post-study quizzes with $B_i$ 4, but the difference in performance gain between groups was not statistically significant ( $B_i$ 5). NeuroVase’s strongest empirical advantage was usability and engagement: SUS was $B_i$ 6, the mobile application pleasantness rating was $B_i$ 7, and engagement was $B_i$ 8 (Jahani et al., 31 Mar 2026).

Adjacent immersive work reinforces similar pedagogical themes. SONIA, a customizable VR platform for brain networks and functional neuroanatomy, uses a multi-scale interaction paradigm with a large immersive brain and a smaller mirrored interactive brain. Its anxiety-network case study integrates subsystem narratives, graph-like connectivity selection, and progress tracking. Although SONIA is VR rather than AR, it sits on the same neuroimmersive continuum and highlights concerns highly relevant to NeuroAR: educational agency, information density, and the tradeoff between immersive context and interface complexity. Its SUS was $B_i$ 9, overall learning yield was rated $s_w = \frac{w_i}{w_m}$ 0, but the multi-scale strategy’s benefit for spatial understanding was essentially neutral at $s_w = \frac{w_i}{w_m}$ 1 (Hellum et al., 2023). A plausible implication is that immersive neurovisualization systems are often limited less by rendering capability than by curriculum design and UI load.

The educational literature also shows that strong usability does not automatically imply superior objective learning gain relative to traditional materials. NeuroVase’s cue-card recognition glitches and SONIA’s crowding complaints illustrate a recurrent implementation issue: the pedagogical promise of NeuroAR depends on tracking stability and restrained interface complexity, not only on 3D visualization quality (Jahani et al., 31 Mar 2026, Hellum et al., 2023).

7. Autoregressive meanings, misconceptions, and open problems

Outside the AR-visualization literature, NeuroAR also denotes autoregressive neuroimaging models. Sparse-plus-low-rank autoregressive identification in neuroimaging time series models the inverse spectrum of the observed process as $s_w = \frac{w_i}{w_m}$ 2, with $s_w = \frac{w_i}{w_m}$ 3 sparse and $s_w = \frac{w_i}{w_m}$ 4 low-rank, and solves the resulting problem with ADMM at a per-iteration complexity of $s_w = \frac{w_i}{w_m}$ 5. On resting-state fMRI from 90 brain regions in 17 subjects, the dynamic low-rank structure recovered latent components aligned with canonical networks such as the visual network, DMN, and ECN, and the authors interpret these components as frequency-dependent spatio-temporal generalizations of classical component analysis (Liégeois et al., 2015). This is conceptually distinct from AR overlays but relevant to the broader semantic range of NeuroAR.

The title-specific model “NeuroAR” in brain aging synthesis is another distinct usage. Here the task is longitudinal MRI prediction, $s_w = \frac{w_i}{w_m}$ 6, using multi-scale discrete token maps from an MS-VQVAE and an autoregressive transformer conditioned on acquisition age and target age through AdaLN and cross-attention. On ADNI, PPMI, and ABCD, NeuroAR reported PSNR/SSIM of 21.08/0.837, 21.63/0.827, and 22.60/0.836, respectively, outperforming latent diffusion and latent StarGAN baselines; synthetic augmentation also improved downstream age prediction on ABCD from MAE 0.685 and $s_w = \frac{w_i}{w_m}$ 7 to MAE 0.582 and $s_w = \frac{w_i}{w_m}$ 8 (Yesiloglu et al., 29 Jul 2025). The term therefore cannot be interpreted as exclusively AR-based.

Several open problems recur across the AR-centered branch. First, one-number accuracy claims are inadequate: the most rigorous work insists on layered measurement that separates tracking-device accuracy, CT-derived ground truth, registration quality, user perception, and task-phase-specific interaction effects (Bagheri et al., 2023). Second, markerless systems remain context-limited when the face is not visible or segmentation quality degrades (Ho et al., 2024). Third, low-cost optical systems trade affordability for susceptibility to lighting variation, marker occlusion, rapid motion, and calibration errors, even when multi-camera fusion mitigates some failures (Hu et al., 28 Jan 2026). Fourth, long-term tracking drift remains under-characterized; limited-window phantom experiments do not yet establish the stability needed for prolonged procedures (Bagheri et al., 2023).

The literature therefore supports a restrained conclusion. NeuroAR is not a single platform, device class, or benchmark. It is a research area organized around in-situ neurospatial representation, spanning surgical guidance, stimulation, education, cognitive support, and neural-interface feedback, with separate but important autoregressive usages in neuroimaging. Its most durable technical lesson is that clinically or cognitively meaningful performance depends less on the mere presence of a hologram than on registration fidelity, feedback structure, and the end-to-end coupling between perception, action, and measurement (Fischer et al., 14 Aug 2025, Wang et al., 25 Sep 2025).