Mobile Eye Tracker Overview

• Mobile eye tracking is a technology that estimates and records real-time gaze using wearable devices equipped with sensors and cameras.
• It leverages diverse methods like IR-based VOG, appearance-based algorithms, and EOG to enhance accuracy and usability in various applications.
• Recent advances include machine learning integration, sensor fusion, and privacy-preserving designs to ensure robust performance in real-world settings.

A mobile eye tracker is a system designed to estimate and record the visual gaze or eye movements of a user in real time, utilizing wearable or hand-held consumer devices such as head-mounted displays (HMDs), smart glasses, smartphones, or tablets. Modern mobile eye trackers leverage a combination of optics, cameras, inertial sensors, dedicated processing units, and machine learning algorithms to deliver precise gaze estimation outside laboratory settings, enabling applications ranging from human-computer interaction (HCI) to cognitive assessment and collective behavior analysis.

1. System Architectures and Sensing Modalities

Mobile eye trackers are architected to balance form factor, power consumption, data privacy, and real-time performance. Key sensing paradigms include:

• Active Infrared Video-Oculography (VOG): Utilized in commercial systems and HMDs (e.g., Apple Vision Pro, Pupil Labs Neon). Infrared LEDs illuminate the eye, and IR-sensitive cameras capture reflections and dark-pupil imagery, enabling robust pupil segmentation and gaze-vector estimation via geometric models or machine learning (Mehmedova et al., 17 Aug 2025, Saxena et al., 8 Jul 2024, Barkevich et al., 28 Mar 2024).
• Appearance-Based Methods: Rely on visible-light RGB cameras (e.g., smartphone front camera) and deep CNNs for end-to-end gaze regression. These enable eye tracking without dedicated hardware, at the cost of increased sensitivity to environmental conditions (Gunawardena et al., 13 Jun 2025, reddy et al., 2023, Davalos et al., 27 Aug 2025).
• Electrooculography (EOG): Employs contact or hybrid contactless electrodes embedded in glasses to capture corneo-retinal potential changes, offering ultra-low power and privacy safety in exchange for coarser spatial precision (Schärer et al., 19 Dec 2024).
• Event Cameras: Asynchronous dynamic vision sensors (DVS) supporting >10 kHz update rates for microsaccade and tremor studies in mobile HMDs (Angelopoulos et al., 2020).
• Magnetic Tracking: Scleral lens-embedded dipoles with head-mounted magnetoresistive sensor arrays, reconstructing 3D gaze and vestibulo-ocular reflex (VOR) characteristics, particularly in clinical diagnostics (Bevilacqua et al., 2023).
• Lensless Optics (FlatCam): Binary mask-based imaging and accelerator co-design, reducing hardware thickness and bandwidth while supporting ROI-focused DNN processing (You et al., 2022).

Architectural Examples

System/Method	Primary Sensor	Platform
iTrace	IR cameras + HMD	Apple Vision Pro
MobileEYE	RGB camera + MobileNetV3	Smartphone/Tablets
WebEyeTrack	Webcam + BlazeGaze CNN	Web browser
ElectraSight	EOG electrodes + TinyML	Smart Glasses
Event-based DVS	Event camera	HMD/Glasses

2. Gaze Estimation Pipelines and Algorithms

Gaze estimation pipelines in mobile trackers are contingent on both sensor type and target platform constraints. Key pipeline components include:

• Eye Region Extraction: Using facial landmark detectors or mesh models (e.g., MediaPipe Face Mesh, ARKit) to localize eyes and normalize for head pose (Davalos et al., 27 Aug 2025).
• Image Preprocessing: Cropping, resizing, normalization, and color conversion; event-based pipelines emphasize delta intensity (Feng et al., 2022, Angelopoulos et al., 2020).
• Feature Extraction and Regression:
  • Model-based (geometric): Fit ellipse or 3D eyeball models to segmented pupil/iris contours, followed by polynomial or direct ray-tracing mapping to scene coordinates. Typical methods use affine or polynomial calibration, or head- and eye-tracker intrinsic models (Barkevich et al., 28 Mar 2024).
  • End-to-end (learning-based): CNNs (e.g., BlazeGaze, MobileNetV3), sometimes augmented with LSTM/ConvLSTM modules for temporal features, directly regress from image patches (with or without head pose) to screen-space gaze points (Davalos et al., 27 Aug 2025, Gunawardena et al., 13 Jun 2025).
• ROI Prediction: Event-driven ROI detectors or co-designed DNN modules, cropping to salient eye regions to minimize computation (Feng et al., 2022, You et al., 2022).
• Calibration: Explicit (dot targets), implicit (saliency-driven or opportunistic), few-shot on-device meta-learning or none (EOG, robust CNNs) (Yang et al., 2022, Davalos et al., 27 Aug 2025, Schärer et al., 19 Dec 2024).
• Personalization: Last-layer fine-tuning (MLP/SVR), MAML adaptation, or fully calibration-free execution.

Gaze Estimation Equations

• Click-based rate: $r = \frac{N_{\rm clicks}}{T_{\rm session}}$
• Normalized 2D screen projection: $u_n = x_n W, v_n = y_n H$ (Mehmedova et al., 17 Aug 2025)
• Polynomial mapping: $\begin{pmatrix}g_x\ g_y\end{pmatrix} = \sum_{0\le i+j\le3} \begin{pmatrix}p_{ij}\q_{ij}\end{pmatrix} u^i v^j$ (Barkevich et al., 28 Mar 2024).

3. Evaluation Metrics and Benchmarks

Multiple accuracy, precision, efficiency, and robustness metrics are standard in mobile eye-tracker evaluation:

• Point of Gaze (PoG) Error: Euclidean distance (mm or cm) between predicted and ground-truth gaze, e.g., 2.32 cm on GazeCapture for WebEyeTrack (Davalos et al., 27 Aug 2025), 17.76 mm for MobileEYE (Gunawardena et al., 13 Jun 2025), sub-degree angular error for event-based and lensless hardware (Angelopoulos et al., 2020, You et al., 2022).
• Precision: Dispersion of gaze samples within fixations; typically reported in degrees of visual angle (Barkevich et al., 28 Mar 2024).
• Dropout Rate: Fraction of samples with error exceeding a threshold (e.g., >10°) (Barkevich et al., 28 Mar 2024).
• Latency: End-to-end system lag, e.g., sub-2.4 ms inference on iPhone 14 for WebEyeTrack, and 301 μs inference for ElectraSight TinyML model (Davalos et al., 27 Aug 2025, Schärer et al., 19 Dec 2024).
• Click Rate / Sampling Rate: For discrete or event-based systems (e.g., 14.22 clicks/s with controller input on iTrace) (Mehmedova et al., 17 Aug 2025).
• Energy/PWR: Continuous operation at <9 mW for EOG (Schärer et al., 19 Dec 2024), <155 mW for FlatCam EyeCoD (You et al., 2022).
• Robustness to Covariates: Sensitivity to lighting, head pose, demographic characteristics, device model, occlusion, and slippage (Gunawardena et al., 13 Jun 2025, Barkevich et al., 28 Mar 2024).

Notable trade-offs include high-precision at the cost of calibration effort (model-based HMD systems), higher energy and processing demand for video pipelines, and privacy/accuracy trade-offs in EOG or optics co-design.

4. Applications and Use Cases

The current generation of mobile eye trackers enables a broad spectrum of domain-specific applications:

• Human–Computer Interaction: Attention-aware user interfaces, gaze-based authentication, mobile UI usability testing, and in-app user engagement analytics (reddy et al., 2023, Mehmedova et al., 17 Aug 2025).
• Education and Skill Assessment: Quantifying instructional focus, reading comprehension, or problem-solving strategies via dynamic heatmaps (Mehmedova et al., 17 Aug 2025).
• Social and Group Behavior: Multi-person eye tracking with coordinated spatio-temporal metrics, enabling large-scale studies of collaborative attention (e.g., concert/film analytics using SocialEyes) (Saxena et al., 8 Jul 2024).
• Marketing and Environmental Analysis: Heatmaps to track ad or product attention in real-world settings (Mehmedova et al., 17 Aug 2025).
• Clinical and Cognitive Science: Cognitive workload, fatigue, impairment assessment, vestibulo-ocular reflex testing, and neurology diagnostics using both camera-based and magnetic/EOG modalities (Schärer et al., 19 Dec 2024, Bevilacqua et al., 2023, Bækgaard et al., 2015).
• Augmented/Virtual Reality: High-throughput, low-latency gaze for foveated rendering, AR/VR control, and mixed-reality content interactivity (You et al., 2022, Feng et al., 2022).
• Accessibility: Gaze-based input for users with motor disabilities and assistive communication scenarios (reddy et al., 2023, Schärer et al., 19 Dec 2024).

5. Methodological Developments and Innovations

Recent work in mobile eye tracking advances methodology along several axes:

• Privacy-Preserving Architectures: Strict on-device computation pipelines with no transmission of face/eye images to server infrastructure. Approaches include WebEyeTrack (browser-local), ElectraSight (bioelectric), and lensless FlatCam (non-reconstructible multiplexed sensing) (Davalos et al., 27 Aug 2025, Schärer et al., 19 Dec 2024, You et al., 2022).
• Calibration and Personalization: Shift from explicit, multiple-point calibration to opportunistic, saliency-aware, or few-shot meta-learning paradigms reducing user friction while maintaining sub-cm accuracy (Yang et al., 2022, Davalos et al., 27 Aug 2025).
• Collective and Unstructured Analysis: NMF-based clustering of Areas of Interest (AOIs) in head-worn video, joint-scene gaze fusion using homography (for group dynamics), and automated AOI labeling in egocentric video with YOLOv2/OpenPose pipelines (Klötzl et al., 4 Apr 2024, Saxena et al., 8 Jul 2024, Callemein et al., 2020).
• Hardware/Algorithm Co-Design: Joint optimization across sensing, DNN architecture, memory hierarchy, and on-chip acceleration (e.g., EyeCoD) to sustain real-time throughput in strict size/power budgets (You et al., 2022).
• Saliency-Leveraged Calibration: Using bottom-up/top-down saliency to bootstrap spindle–free calibration and automatically exploit meaningful content frames for parameter refinement (Yang et al., 2022).

6. Limitations and Challenges

Modern mobile eye trackers face several unresolved issues:

• Data Access and Privacy Restrictions: Systems like Apple Vision Pro only expose gaze coordinates at discrete events, necessitating workarounds (e.g., click-based sampling as in iTrace) (Mehmedova et al., 17 Aug 2025).
• Environmental Sensitivity: Appearance-based and RGB methods show marked error increases with poor lighting, occlusions, or aging demographics. IR-based and EOG systems are more resilient but introduce cost, calibration, or invasiveness (Gunawardena et al., 13 Jun 2025, Schärer et al., 19 Dec 2024).
• Real-World Variability: Field deployment must address blinks, head/eye slippage, glasses reflections, and calibration drift; domain adaptation and transfer learning are active areas (reddy et al., 2023, Bækgaard et al., 2015).
• Power and Bandwidth: Ultra-low-power modalities (EOG, magnetic) trade spatial resolution, while high-throughput camera pipelines require power or thermal budget optimization.
• Scalability and Open Source: Many recent solutions emphasize open code and reproducible pipelines (WebEyeTrack, Open Gaze, SocialEyes), but heterogeneous hardware and privacy policies still constrain global adoption (Davalos et al., 27 Aug 2025, reddy et al., 2023, Saxena et al., 8 Jul 2024).

7. Future Directions

Research trajectories in mobile eye tracking include:

• True 3D/Spatial Gaze Mapping: Integration of depth imaging, SLAM, and structure-from-motion to translate gaze coordinates into global metrics for AR and robotics (Mehmedova et al., 17 Aug 2025).
• Domain Adaptivity: Unsupervised transfer to new users, devices, and environments via meta-learning, adversarial adaptation, and temporal model integration (Davalos et al., 27 Aug 2025, reddy et al., 2023).
• Sensor Fusion: Hybridizing EOG, IR video, and inertial sensing for drift compensation, blink recovery, and robustness.
• Privacy-First Design: Ensuring continuous eye-based interaction without risk of data leakage or facial exposure, including in hardware-level encryption and non-imaging modalities (Schärer et al., 19 Dec 2024, You et al., 2022).
• Collective and Social Interaction Analytics: Scalable multi-person pipelines, online group attention metrics, and real-time gaze-based content feedback for live events or collaborative HCI (Saxena et al., 8 Jul 2024).
• Miniaturization and Wearability: Advancements in lensless imaging, soft EOG, and wireless power to achieve ubiquitous “invisible” gaze tracking in daily life (Schärer et al., 19 Dec 2024, You et al., 2022, Bevilacqua et al., 2023).

Mobile eye tracking as a field is rapidly evolving, with rigorous comparative benchmarks and methodological innovations being actively developed. Robust, privacy-respecting, and energy-efficient systems deployed in naturalistic settings are now enabling scientific and practical advances once limited to controlled laboratory experiments. In all settings, system selection and deployment must consider accuracy, robustness, power, privacy, and long-term usability (Davalos et al., 27 Aug 2025, Mehmedova et al., 17 Aug 2025, Schärer et al., 19 Dec 2024, You et al., 2022, Saxena et al., 8 Jul 2024, Gunawardena et al., 13 Jun 2025, reddy et al., 2023).