NeuroGaze: Gaze-Based Control and Estimation

• NeuroGaze is a diverse family of gaze-centered systems that employ low-cost sensors, deep learning, and multimodal fusion to enable both assistive and interactive human-computer interfaces.
• It integrates various methodologies, from appearance-based CNN regressions and geometry-driven models to hybrid EEG and eye-tracking pipelines, to accurately interpret gaze signals.
• Key applications include assistive technology for motor-impaired users, attention analysis in educational settings, and robotic as well as VR interfaces, highlighting its practical and scalable design.

NeuroGaze denotes a family of gaze-centered systems, models, and frameworks rather than a single canonical architecture. Across the literature, the term encompasses low-cost webcam interfaces for cursor control, appearance-based RGB gaze estimation, gaze-and-EEG pipelines for attention analysis, hybrid EEG+eye-tracking interfaces in virtual reality, robotics-oriented gaze control, and even sensorless perceptual mechanisms that induce mutual gaze without cameras or actuators (Urumkar et al., 2023, Herashchenko et al., 2023, Xu et al., 2017, Coutray et al., 9 Sep 2025). The unifying theme is the operational use of gaze, gaze-related physiology, or gaze perception as a control variable, an inference target, or a socially meaningful signal.

1. Scope and nomenclature

In the surveyed literature, “NeuroGaze” functions as an umbrella label for technically distinct research programs. Some papers use the name directly for a concrete system; some explicitly frame a method as “NeuroGaze” for clarity; and one webcam-based interface is officially named “NeuGaze,” with the note that NeuGaze and NeuroGaze refer to the same system (Yang, 21 Apr 2025, Tabatabaei et al., 11 Feb 2026).

Usage	Modality	Objective
Low-cost cursor controller (Urumkar et al., 2023)	Webcam, dlib, OpenCV, PyAutoGUI	Eye-gesture computer control
Appearance-based estimator (Herashchenko et al., 2023)	RGB camera, RetinaFace, 6DRepNet, CNN	Pitch/yaw gaze regression
Audience-attention pipeline (Xu et al., 2017)	Eye tracking, EEG, TWED, FMSCD	Attention inference in lectures
Hybrid VR BCI (Coutray et al., 9 Sep 2025)	Eye tracking + EEG	Hands-free target selection
Sensorless HRI mechanism (Kadem, 10 Apr 2026)	Concave eye sockets, convexity prior	Mutual gaze without sensors

This breadth is not merely terminological. It reflects different definitions of what counts as a gaze system: direct eye-based control, screen-point regression, 3D scene-conditioned attention fields, EEG-based estimation of gaze variables, robot gaze behavior prediction, and perceptual engineering of apparent eye contact. A plausible implication is that NeuroGaze is best understood as a research area organized around gaze as an interaction primitive rather than as a fixed software stack.

2. Assistive and low-cost human–computer interaction

A central NeuroGaze lineage is low-cost assistive control for people with severe motor impairments. In “Control of Computer Peripherals using Human Eyes,” NeuroGaze is a low-cost, webcam-based eye-gaze cursor controller intended for users with ALS, multiple sclerosis, brainstem stroke, cerebral palsy, spinal cord injury, and paraplegia, and it targets tasks such as email, web browsing, and media playback (Urumkar et al., 2023). Its pipeline is explicit: face detection → eye-region isolation → pupil localization → blink detection and gaze direction estimation → gesture recognition → cursor and action mapping. The implementation uses the dlib frontal face detector, dlib’s pre-trained facial landmark shape predictor, OpenCV image processing, and PyAutoGUI. Within the eye ROI, it applies bilateral smoothing, erosion, thresholding, contour finding, and centroid computation; blink detection uses landmark-derived horizontal and vertical lines and the logic “Ratio > Threshold ⇒ Blink,” “Ratio ≤ Threshold ⇒ No Blink.” Control is discrete rather than continuous: look left/right/up/down/center drives directional cursor motion, and blink triggers a click.

That design is deliberately pragmatic. It avoids corneal reflections, IR illumination, and wearable hardware; calibration is intentionally minimized; and robustness is sought through morphological filtering and per-frame geometric normalization with $H$ and $V$. At the same time, the paper is explicit about limits: no quantitative metrics are reported, head-pose compensation is not modeled explicitly, and performance is bounded by visible-light contrast, reflections, eyelid occlusion, glasses, threshold drift, and blink fatigue. The result is a system optimized for a small, reliable gesture set rather than precise point-of-gaze regression.

A more expansive webcam-only variant appears in NeuGaze, which uses a standard 30 Hz webcam, Mediapipe, L2CS-Net, and a scikit-learn LASSOCV regression to combine gaze, head movements, and facial expressions for cursor navigation, key triggering, and dynamic gaming interactions (Yang, 21 Apr 2025). Mediapipe supplies 468 facial landmarks, 52 blend shapes, and pitch, yaw, and roll; L2CS-Net predicts gaze angles; and LASSOCV maps $$[ \text{gaze angles}, \text{face box coordinates} ]$$ to screen coordinates. The system supports absolute and relative cursor modes and compresses command vocabularies with a “skill wheel.” The reported evidence is qualitative and single-user, but it extends the assistive NeuroGaze idea from eye-only interaction to a multimodal neck-up control scheme.

3. Appearance-based and geometry-based gaze estimation

A second major usage of NeuroGaze concerns continuous gaze estimation from commodity RGB cameras. In “Appearance-based gaze estimation enhanced with synthetic images using deep neural networks,” NeuroGaze is a modular pipeline built from RetinaFace, 6DRepNet, and a custom lightweight CNN regressor that predicts absolute eye pitch and yaw in the camera frame from concatenated eye crops (Herashchenko et al., 2023). RetinaFace detects the face and five landmarks, 6DRepNet estimates head pose, and left/right eye crops are concatenated horizontally into a $100 \times 300$ RGB input after an initial $300 \times 900$ design. The final CNN has 2,092,342 trainable parameters; training uses MSE, evaluation uses MAE, and head-pose features are concatenated into the last hidden layer, improving MAE by approximately $0.2^\circ$. On Columbia Gaze under four-fold cross-validation, the reported mean absolute error is $1.93^\circ \pm 1.50^\circ$, and inference runs at about 30 FPS on a notebook GPU. The paper also introduces a 57,375-image MetaHuman Synthetic Dataset, and the combined training setup is described as the most stable across domains.

The same article makes clear that in-domain accuracy and cross-domain robustness are different problems. Columbia $\rightarrow$ Columbia yields $1.93^\circ \pm 1.50^\circ$, but Columbia $\rightarrow$ MetaHuman gives $V$0, and MetaHuman $V$1 Columbia gives $V$2. This domain gap motivates synthetic–real combination, head-pose conditioning, and future domain adaptation. The NeuroGaze formulation here is therefore not merely “RGB gaze estimation,” but RGB gaze estimation with explicit attention to generalization, calibration minimization, and practical deployment on standard cameras.

A complementary approach is purely geometry-based. “Low-cost Geometry-based Eye Gaze Detection using Facial Landmarks Generated through Deep Learning” derives an 8-dimensional manifold from MediaPipe Attention Mesh outputs: $V$3, where the descriptors combine binocular pupil position, head-rotation proxies, head position, and face scale (Ye et al., 2023). A linear model $V$4 maps this descriptor to gaze angles or screen coordinates. The system uses 468 facial landmarks and iris landmarks, runs the landmark network in about 16.6 ms per frame, and reports mean angular error below $V$5, specifically $V$6 on $V$7 and $V$8 on $V$9, with $$[ \text{gaze angles}, \text{face box coordinates} ]$$0 on $$[ \text{gaze angles}, \text{face box coordinates} ]$$1 and $$[ \text{gaze angles}, \text{face box coordinates} ]$$2 on $$[ \text{gaze angles}, \text{face box coordinates} ]$$3, and distance error $$[ \text{gaze angles}, \text{face box coordinates} ]$$4 cm on screen. Relative to the appearance-based NeuroGaze variant, this geometry-driven formulation trades deep feature learning for an interpretable descriptor and a short per-user calibration.

A third reference point is the hardware-agnostic RGB tracker of “Towards Hardware-Agnostic Gaze-Trackers,” which presents a four-branch appearance-based CNN with left eye, right eye, face crops, and a $$[ \text{gaze angles}, \text{face box coordinates} ]$$5 face-grid (Sharma et al., 2020). Using ResNet18, Dlib-based detection, rotation correction, YCbCr inputs, batch normalization, dropout, and cyclical learning rate, the reported test error on GazeCapture is 1.8073 cm without any calibration or device specific fine-tuning. This establishes a distinct NeuroGaze trajectory: calibration-free, device-agnostic screen-point regression on ordinary RGB cameras.

4. EEG, attention inference, and hybrid brain–gaze interfaces

NeuroGaze also appears in systems where gaze is inferred, structured, or confirmed through electrophysiological signals. In “Report: Dynamic Eye Movement Matching and Visualization Tool in Neuro Gesture,” NeuroGaze is a computational pipeline and interactive tool for audience attention analysis during lectures (Xu et al., 2017). It combines fixation time series, synchronized 64-channel EEG, Time Warp Edit Distance (TWED), fast multi-scale community detection, and a PyQt5 visualization tool. The operational definition of attention is high inter-subject gaze alignment within a time window, optionally corroborated by EEG. Eye-tracking data are smoothed with an 80 ms triangular window, viewers with $$[ \text{gaze angles}, \text{face box coordinates} ]$$6 missing samples are excluded, remaining missing data are imputed via sparse PCA plus linear interpolation, and pairwise gaze-trajectory similarity is computed with TWED. Practical exploration led to $$[ \text{gaze angles}, \text{face box coordinates} ]$$7 and $$[ \text{gaze angles}, \text{face box coordinates} ]$$8 as visually robust defaults. Here NeuroGaze does not control a cursor or regress a screen point; it estimates collective attentional structure from gaze synchrony.

A more direct EEG-based formulation appears in “An Interpretable and Attention-based Method for Gaze Estimation Using Electroencephalography,” where NeuroGaze is an interpretable regression model from EEG to gaze variables (Weng et al., 2023). Using EEGEyeNet, 128-channel EEG at 500 Hz, and 1-second clips, the model applies 12 convolution blocks with Squeeze-and-Excitation and Self-Attention modules to estimate either absolute gaze position $$[ \text{gaze angles}, \text{face box coordinates} ]$$9 or saccade amplitude and direction. On the Position Task, CNN + SE reaches $100 \times 300$0 px $100 \times 300$1; on the Direction Task, CNN + SE + SA reaches angle RMSE $100 \times 300$2 and amplitude RMSE $100 \times 300$3 px. The paper emphasizes interpretability: noisy electrodes around the ears are down-weighted, and across samples with at least one noisy electrode in the Direction Task, 42% of noisy electrodes had normalized attention weights below 0.05 versus 19% for non-noisy electrodes. NeuroGaze in this sense is a neurophysiological decoder, not an optical gaze tracker.

A hybrid consumer BCI formulation is presented in “NeuroGaze: A Hybrid EEG and Eye-Tracking Brain-Computer Interface for Hands-Free Interaction in Virtual Reality” (Coutray et al., 9 Sep 2025). The system combines Meta Quest Pro eye tracking at 72 Hz with Emotiv EPOC X EEG and uses a late-fusion gating strategy: gaze designates a target and an EEG “pull” mental command confirms selection. In a 360° cube-selection task, completion times were $100 \times 300$4 s for VR controllers, $100 \times 300$5 s for gaze+pinch, and $100 \times 300$6 s for NeuroGaze; error rates were $100 \times 300$7, $100 \times 300$8, and $100 \times 300$9, respectively. Workload results showed lower Physical Demand for NeuroGaze than for controllers, but mixed overall preferences. The paper explicitly frames the result as a speed–accuracy tradeoff: fewer errors and lower physical demand, but substantially longer completion times.

5. Robotics, 3D environments, and gaze redirection

In embodied systems, NeuroGaze frequently means the integration of gaze with scene geometry, motor planning, or robot social behavior. “Free-View, 3D Gaze-Guided, Assistive Robotic System for Activities of Daily Living” combines SMI eye-tracking glasses, Kinect v2 RGB-D sensing, ORK/LINEMOD object recognition, MoveIt!, and a UR5 manipulator to estimate a 3D point of regard under free head movement and translate fixations into robot actions (Wang et al., 2018). The reported average Euclidean 3D PoR error is $300 \times 900$0 cm, fixation computation time is $300 \times 900$1 s, gaze-guided object recognition success is 98.67%, path planning success is 96.67%, overall system success rate is 86.67%, and manual pick-and-place achieved a success rate of 100% on the users’ first attempt. This NeuroGaze variant treats gaze as a world-anchored command signal for assistive manipulation.

“Developing Neural Network-Based Gaze Control Systems for Social Robots” frames a sequence-modeling approach as NeuroGaze for clarity (Tabatabaei et al., 11 Feb 2026). Here the problem is not estimating where a human looks, but predicting who a robot should look at next in multi-party social interaction from recent sequences of social cues. Two-layer LSTMs and compact Transformers operate on per-person feature tensors derived from 2D animation and 3D VR scenarios. Reported top-1 performance is about 60% in the 2D animation and about 65% in the 3D animation, and the best model was implemented on a Nao robot and evaluated by 36 participants. This extends NeuroGaze from perceptual estimation to social gaze control and motion-time pattern learning.

Three-dimensional scene-conditioned and generative models broaden the term further. “Neural Radiance and Gaze Fields for Visual Attention Modeling in 3D Environments” extends a pre-trained NeRF with a gaze field network to produce pixel-wise salience maps conditioned on an arbitrary observer pose decoupled from the render camera, with explicit occlusion-aware “gaze shadowing” driven by NeRF density (Chubarau et al., 10 Mar 2025). “NeRF-Gaze: A Head-Eye Redirection Parametric Model for Gaze Estimation” uses a head–eye redirection parametric model based on Neural Radiance Fields to generate dense, view-consistent gaze data and improve domain generalization and adaptation (Yin et al., 2022). “Roll Your Eyes: Gaze Redirection via Explicit 3D Eyeball Rotation” replaces implicit NeRF-style eye control with an explicit 3D eyeball structure represented by 3D Gaussian Splatting; on ETH-XGaze it reports, for the facial area, SSIM 0.905, PSNR 30.292, LPIPS 0.062, FID 31.272, gaze 5.006, and identity 91.947 (Choi et al., 8 Aug 2025). In these works, NeuroGaze becomes a family of 3D-aware rendering, synthesis, and redirection methods rather than a real-time interaction frontend.

A conceptually extreme case is “Perception Is All You Need: A Neuroscience Framework for Low Cost Sensorless Gaze in HRI,” which describes a sub-dollar cardboard robot with concave eye sockets and painted pupils that appears to make eye contact through the convexity prior, the hollow-face illusion, predictive processing, and STS-mediated gaze computation (Kadem, 10 Apr 2026). No camera detects the viewer; no servomotor rotates the eye; the viewer’s brain is effectively the actuator. This use of NeuroGaze shifts the locus of computation from the machine to human perception itself.

6. Scientific foundations, limitations, and recurrent debates

Several foundational works clarify why many NeuroGaze systems emphasize eye regions, head–eye coupling, and task-specific training. “Measuring and modeling the perception of natural and unconstrained gaze in humans and machines” shows that humans do nearly as well when only an eyes-region strip is visible as when the whole face is visible, and that a two-stage computational model—head orientation first, then eyes-conditioned gaze—replicates the human pattern and reproduces the Wollaston illusion when trained for gaze rather than recognition (Harari et al., 2016). This directly supports later designs that crop eyes narrowly, inject head pose as auxiliary information, or decompose gaze estimation into conditional stages.

Neurophysiological syntheses complicate any overly simple reading of “gaze decoding.” “Neurophysiology of gaze orientation: Core neuronal networks” describes the brainstem–cerebellar circuitry underlying rapid, precise gaze shifts, including premotor burst generators, omnipause neurons, neural integrators, cerebellar fastigial and floccular contributions, and the decomposition $300 \times 900$2 during eye–head coordination (Goffart et al., 2023). “Orienting gaze toward a visual target: Neurophysiological synthesis with epistemological considerations” goes further and argues against a one-to-one correspondence between measured physical values of gaze or head orientation and neuronal activity, emphasizing instead “poly-equilibrium” and balances across antagonistic visuomotor channels (Goffart, 2024). A common misconception is therefore that all NeuroGaze systems are simply estimating a stable, directly encoded variable called “gaze.” The neurophysiological literature suggests a distributed, context-sensitive control problem rather than a single scalar latent.

Across implementations, several limitations recur. Webcam and visible-light systems are sensitive to illumination, reflections, occlusions, dark irises, and large head movements; some gesture-based interfaces report no quantitative evaluation at all (Urumkar et al., 2023). Appearance-based deep models still exhibit strong synthetic-to-real or cross-dataset gaps, even when in-domain accuracy is strong (Herashchenko et al., 2023). Hybrid EEG+gaze interfaces reduce physical demand and unintended activation, but currently incur substantial latency and a clear speed–accuracy tradeoff in immersive tasks (Coutray et al., 9 Sep 2025). This suggests that future NeuroGaze systems will continue to depend on adaptive thresholds, temporal modeling, richer synthetic variation, domain adaptation, low-power hardware integration, and accessibility-aware UI design rather than on any single breakthrough component.

In aggregate, NeuroGaze identifies a broad technical domain in which gaze is treated as a computational signal, a behavioral proxy, a control channel, a rendered attribute, or a perceptual effect. Its internal diversity is unusually high, but the literature converges on a consistent set of themes: low-cost sensing, head–eye context, multimodal fusion, 3D scene awareness, and the recognition that gaze is both an engineering variable and a neurocognitive phenomenon.