EyeMulator: Multifunctional Ocular Emulation

• EyeMulator is a term applied to multiple, distinct systems that emulate ocular data, including vision simulation, gaze movement generation, and attention-based fine-tuning in CodeLLMs.
• Its implementations range from VR-based infrared periocular expression inference with one-shot user normalization to probabilistic eye-movement and noise-modeled gaze trajectory simulations.
• Across applications, EyeMulator systems enhance performance metrics, enable synthetic hardware prototyping, and facilitate advanced computational models by integrating human visual-attention statistics.

Searching arXiv for "EyeMulator" and closely related usages to ground the article in the cited literature. EyeMulator is not a single canonical system in the arXiv literature. The name has been used for multiple technically distinct constructs centered on ocular sensing, eye-movement generation, vision simulation, and gaze-informed learning. In the cited works, it denotes an algorithm for inferring upper-face expressions from infrared eye-tracking cameras inside VR headsets and driving expressive avatars in real time; a parametric simulator of fixations, saccades, smooth pursuits, and two-dimensional gaze traces; an end-to-end framework for prototyping eye-tracker hardware from synthetic 3D eye data; ophthalmic simulators for intraocular-lens evaluation and multi-depth near-eye display; and a fine-tuning method for CodeLLMs that injects human visual-attention priors into the loss function (Hickson et al., 2017, Fuhl et al., 2018, Fuhl et al., 2018, Lin et al., 20 Mar 2025, Akyazi et al., 2023, Zolfaghari et al., 22 Dec 2025, Zhang et al., 22 Aug 2025).

1. Terminological scope

Across the cited literature, “EyeMulator” functions as a reused label rather than a unified framework. The commonality is not a shared implementation, but the use of eye-centered data, eye-like optical simulation, or attention patterns derived from gaze.

Usage	Core function	arXiv id
VR expression inference	Infer facial expressions from IR eye images and drive avatars	(Hickson et al., 2017)
Eye-movement simulation	Generate fixation, saccade, pursuit, and gaze traces; in one variant, train detectors	(Fuhl et al., 2018, Fuhl et al., 2018)
Eye-tracker hardware prototyping	Predict ML gaze-estimation performance from synthetic 3D eye renderings	(Lin et al., 20 Mar 2025)
Ophthalmic simulation	Simulate post-operative vision and assess IOL behavior	(Akyazi et al., 2023, Zolfaghari et al., 22 Dec 2025)
CodeLLM training	Reweight token losses to mimic human visual attention on code	(Zhang et al., 22 Aug 2025)

A common misconception is that EyeMulator names a single eye-tracking toolkit. The literature does not support that interpretation. Instead, the term spans at least five research programs with different objectives, modalities, and evaluation criteria.

2. Infrared periocular expression inference in VR

In the VR line, the EyeMulator approach described in "Eyemotion: Classifying facial expressions in VR using eye-tracking cameras" addresses the fact that head-mounted displays occlude a large portion of the face, blocking facial expressions and thereby restricting social engagement cues among users (Hickson et al., 2017). The system uses two commercial VR headsets, each equipped with a pair of infrared (880 nm) eye-tracking cameras behind the lens via beam splitters. HMD1 captures 200×200 pixels per eye, HMD2 captures 320×240 pixels per eye, and both run at 10 Hz. The dataset comprises 23 users per headset, 46 total, balanced for age, gender, ethnicity, with three HMD sessions per user to capture fit variation.

The label space is explicitly split between ten upper-face Action Units and five emotive expressions. The Action Units are Neutral (AU0), Left Brow Raise (AU1+2L), Right Brow Raise (AU1+2R), Brow Lower (AU4), Upper Lid Raise (AU5), Squint (AU44), Both Eyes Closed (AU43), Left Wink (AU46L), Right Wink (AU46R), and Cheek Raise (AU6). The emotive expressions are Neutral, Anger, Surprise, Happiness, and Eyes Closed as a control. Preprocessing removes blinks with a small “blink vs. not-blink” classifier, rectifies each eye image via a pre-computed calibration map, concatenates the two eye views side-by-side, and resizes the result to 299×299 pixels. Augmentation is restricted to –2% rotation, scale, and brightness, without flipping so that left/right semantics are preserved.

The classifier adopts and fine-tunes Google’s InceptionV3. The input is one 299×299×2 image, the base is pre-trained on ImageNet for 150 k iterations, the Inception modules 3a…5b remain intact, and the head is global average pooling followed by a fully connected layer and a softmax over $C$ classes, with $C=5$ for emotive expressions and $C=10$ for Action Units. Regularization uses L2 weight decay $\lambda = 4e^{-4}$ and no dropout. Optimization uses RMSProp with momentum $0.9$, decay $0.9$, $\epsilon = 1.0$, initial learning rate $0.045$, learning-rate decay by $0.94$ every epoch, batch size $32$, and training until validation loss plateaus at approximately $C=5$0–$C=5$1 epochs.

The distinctive contribution is one-shot per-user normalization. For each user/session, the first 5 s of Neutral eye images $C=5$2 are collected, the mean neutral image is computed as

$C=5$3

and the network receives

$C=5$4

instead of raw $C=5$5. Inter-subject variation in eye shape, brow position, and headset fit was identified as a dominant source of error, and this subtraction boosts accuracy by 5–7% with $C=5$6 (Hickson et al., 2017).

Under 5-fold cross-validation with held-out users, the system reports, for emotive expressions, mean accuracy $C=5$7 and $C=5$8 without personalization, versus mean accuracy $C=5$9 and $C=10$0 with personalization. For Action Units, it reports mean accuracy $C=10$1 and $C=10$2 without personalization, versus mean accuracy $C=10$3 and $C=10$4 with personalization. Advanced human raters on a held-out subset of 350 images achieve accuracy $C=10$5, $C=10$6 without neutral reference, and accuracy $C=10$7, $C=10$8 with neutral reference; the system outperforms both novice and trained human raters by approximately 10 points. The real-time avatar loop—image capture, $C=10$9, CNN forward pass, exponential moving average smoothing with typically $\lambda = 4e^{-4}$0, and rig update—runs under 15 ms on a mobile-grade GPU.

3. Probabilistic eye-movement and gaze-trace simulation

A second usage of EyeMulator denotes a mathematical simulator of eye movements and gaze traces (Fuhl et al., 2018), with a related extension for detector creation (Fuhl et al., 2018). The core pipeline is five-stage: sequence generation of fixations (F), saccades (S), and smooth pursuits (P); continuous velocity-profile generation; re-sampling to arbitrary, possibly time-varying, target sampling rates; user-configurable noise injection; and mapping of the resulting one-dimensional velocity signal into two-dimensional gaze positions on static images or video frames.

For saccades, the instantaneous velocity is modeled by a Gamma-shaped pulse,

$\lambda = 4e^{-4}$1

with shape parameter $\lambda = 4e^{-4}$2 and scale parameter $\lambda = 4e^{-4}$3 (Fuhl et al., 2018). In the detector-oriented formulation, EyeMulator samples a desired skewness $\lambda = 4e^{-4}$4, computes $\lambda = 4e^{-4}$5, sets $\lambda = 4e^{-4}$6, reparametrizes in time over duration $\lambda = 4e^{-4}$7, and optionally adds jitter $\lambda = 4e^{-4}$8 sampled from a Uniform or Normal distribution (Fuhl et al., 2018). For smooth pursuit onset, the simulator uses a logistic rise,

$\lambda = 4e^{-4}$9

or, in the detector paper’s notation,

$0.9$0

followed by either a constant-velocity segment or a linear ramp (Fuhl et al., 2018, Fuhl et al., 2018). Fixations are modeled as low-variance random fluctuations.

Sampling-rate variability is a central design feature. The simulator first generates profiles at a very high internal rate, then draws sampling intervals $0.9$1 from a user-specified Uniform or Normal distribution between $0.9$2 and $0.9$3, and averages the underlying continuous signal over each interval. This explicitly models dropped frames, timestamp jitter, and on-the-fly frame-rate changes. Spatial mapping then integrates the sampled velocity along straight-line bearings between successive targets:

$0.9$4

Targets can be drawn from saliency-map maxima, from real fixation libraries, or from annotated object keypoints; optional scattering yields a 2D Gaussian spread for fixations (Fuhl et al., 2018).

The MATLAB implementation is integrated into EyeTrace and configured by a single MATLAB structure params, including params.sequence, params.fix, params.sac, params.pur, params.sampling, params.noise, and params.mapping (Fuhl et al., 2018). The output gaze is an $0.9$5 array of $0.9$6 degrees of visual angle or pixel coordinates, and t is an $0.9$7 timestamp vector in seconds.

The detector-creation extension replaces manual parameter tuning with a learning pipeline built from randomly generated binary decisions, random ferns, and class-specific strong detectors (Fuhl et al., 2018). Approximately 300 000 candidate binary decisions are generated, the top 10% are retained, approximately 1000 ferns are formed and again the top 10% are kept, and strong detectors are assembled for fixation, saccade, pursuit, and noise classes. On Larsson, GazeCom, and I-BDT data, the proposed method remains more balanced across classes than EV, I-BDT, or LS. The same paper reports median simulator errors of approximately $0.9$8–$0.9$9 $0.9$0 for fixations, approximately $0.9$1–$0.9$2 for saccades, and approximately $0.9$3–$0.9$4 for pursuits. A related validation against four public datasets reports that fixations yield the smallest errors, saccades the largest dispersion of squared error, and pursuit errors peak on I-BDT because its sampling is too sparse to capture the rapid sigmoid onset (Fuhl et al., 2018).

4. Synthetic-data prototyping of eye-tracker hardware

In "Digitally Prototype Your Eye Tracker: Simulating Hardware Performance using 3D Synthetic Data," EyeMulator becomes an end-to-end framework for evaluating how hardware changes impact machine-learning–based eye-tracking performance entirely in silico, with zero new real-world data collection (Lin et al., 20 Mar 2025). The pipeline has four stages: light-dome capture and 3D reconstruction, novel-view rendering via a hybrid NeRF-mesh eye model, optical effects simulation, and machine-learning–based gaze estimation with downstream performance analysis.

The 3D representation combines a static NeRF reconstruction for each of 114 discrete gaze directions per identity with a personalized 3D eyeball mesh that captures corneal refraction and specular glints. The NeRF is parameterized by an MLP $0.9$5 that takes a 3D location $0.9$6 and a view direction $0.9$7 and outputs a volume density $0.9$8 and emitted radiance $0.9$9. Rendering follows volumetric ray marching:

$\epsilon = 1.0$0

with accumulated transmittance

$\epsilon = 1.0$1

Training minimizes a per-pixel photometric loss over calibrated captures (Lin et al., 20 Mar 2025).

Synthetic image generation varies both hardware and nuisance factors. Camera extrinsics are taken from real device calibrations or CAD/optical designs, then perturbed to simulate device fit by $\epsilon = 1.0$2 and $\epsilon = 1.0$3. Intrinsics, including focal length $\epsilon = 1.0$4, are varied over ranges such as $\epsilon = 1.0$5 px. Optical blur is modeled by convolution with a PSF, brightness by $\epsilon = 1.0$6 for $\epsilon = 1.0$7, and sensor degradation by Gaussian read/shot noise plus 8-bit quantization.

Each synthetic dataset is used to train a fresh Project Aria gaze estimator: a ResNet-18 accepting a pair of 240×320 monocular eye crops and outputting a 3D unit gaze vector $\epsilon = 1.0$8 under angular loss

$\epsilon = 1.0$9

The synthetic split is 195 identities, partitioned as 155 train and 40 test, with each identity contributing 114 gazes × 24 slippages, approximately 2.7k images per identity. Training uses Adam, learning rate $0.045$0, batch size $0.045$1, 50 epochs, and three random splits (Lin et al., 20 Mar 2025).

Evaluation uses $0.045$2, $0.045$3, and $0.045$4 percentiles of angular error and the Pearson correlation coefficient $0.045$5 between synthetic and real performance trends as hardware parameters are swept. Reported results are highly specific. In the baseline no-optics setting, synthetic-trained and real-trained models achieve comparable error percentiles on their own domains, with examples including Synth $0.045$6 and Big Aria $0.045$7. As blur radius increases from 0 to 32 px, degradation is nearly identical with $0.045$8 for $0.045$9, $0.94$0, and $0.94$1. For brightness scaling, $0.94$2–$0.94$3, with accurate localization of the saturation/darkness cliff. For Gaussian noise corresponding to PSNR 20–40 dB, $0.94$4. Novel-pose sweeps show monotonic improvement as the camera becomes more on-axis, a peripheral-to-frontal sweep reducing median error from approximately $0.94$5 to approximately $0.94$6, and focal-length variation showing an initial error improvement of approximately 10% before field-of-view becomes too narrow (Lin et al., 20 Mar 2025).

The paper also states clear limits: only 195 scanned persons, gaze range limited to $0.94$7, monocular only with the left eye mirrored from the right, and wavelength fixed at 850 nm.

5. Ophthalmic vision simulation and IOL assessment

A distinct ophthalmic usage couples artificial-eye optics with display engineering to simulate post-operative vision and characterize intraocular lenses (Akyazi et al., 2023), while a later system, Katsim, emphasizes a compact multi-depth near-eye display with synchronized depth coding and pupil-guided eyebox steering (Zolfaghari et al., 22 Dec 2025).

The artificial-eye platform comprises a 3D-printed chamber filled with index-matching fluid $0.94$8, a curved translucent glass retina, an adjustable iris from approximately 2 mm to approximately 6 mm, and an interchangeable IOL holder accommodating standard 6 mm optic diameter IOLs at physiological distances of approximately 5–6 mm from the retina (Akyazi et al., 2023). The realized device was populated with either a 14.5 D monofocal or a 15.5+3.25 D bifocal IOL. The holographic display subsystem uses either a He–Ne laser at $0.94$9 nm or a green LED at $32$0 nm, with a phase-only LCoS SLM of 1920×1080 pixels, 8 µm pixel pitch, 0–$32$1 phase modulation, and 60 Hz update rate. Zemax spot-diagram analysis yields an Airy disk radius of approximately 4.3–4.6 µm on the glass retina, corresponding to a theoretical maximum resolution greater than 100 lp/mm. Experimentally, monofocal IOL contrast preserves more than 80% at 10 cpd, bifocal contrast drops below 50% at the same frequency, and reported visual acuity is approximately 20/22 for the monofocal lens and approximately 20/25 for the bifocal lens (Akyazi et al., 2023).

Katsim replaces holographic beam steering with a time-multiplexed multi-depth architecture based on an amplitude-modulated LCoS SLM, RGB LED illumination, and a high-speed varifocal lens (Zolfaghari et al., 22 Dec 2025). The AM-SLM has 4.5 µm pixel pitch, 1920 × 1080 resolution, and 60 Hz refresh. Each 60 Hz frame is divided into three RGB subframes, so that

$32$2

producing an effective 180 Hz depth-coded cycle. Within each subframe, the SLM is addressed, the varifocal lens settles in less than 2 ms, and the corresponding LED is turned on for the remainder of the subframe. The rendered depth range extends from approximately 0.2 m to optical infinity. The configurable eyebox is 1 to 5 mm, with exit-pupil scaling from 0.85 to 4.7 mm under magnification $32$3, and the system field of view is 9.15 degrees (Zolfaghari et al., 22 Dec 2025).

The Katsim paper also integrates an infrared bright-pupil tracking module. Bright areas correspond to clear crystalline-lens regions and dark areas indicate cataract or opacity. At approximately 60 Hz, the system thresholds the IR image, extracts the largest bright region, computes its centroid, converts the centroid to physical offsets, and mechanically steers the eyebox through the clearest lens region. In both ophthalmic papers, the EyeMulator label denotes preoperative counseling and objective optical evaluation rather than eye tracking or gaze estimation. The limitations are correspondingly optical: coherence-induced speckle and interference fringes in laser-based holography, static benchtop operation in the 2023 system, and the need for patient-specific lens mapping and aberration compensation for clinical translation (Akyazi et al., 2023, Zolfaghari et al., 22 Dec 2025).

6. Human-attention-informed training of code LLMs

In the software-engineering line, EyeMulator is a fine-tuning technique for CodeLLMs that injects token-wise weights derived from human eye-tracking experiments (Zhang et al., 22 Aug 2025). The source corpus is EyeTrans: 27 intermediate-to-advanced Java programmers, mean 20 ± 2 CS semesters, recorded with a monocular eye tracker at 120 Hz and mean angular error less than 0.4°. The raw samples are segmented by an I-DT algorithm into fixations of at least 100 ms within a 1° dispersion window and saccades of 40–50 ms; blanks, blinks, and noise outliers are removed; approximately 3% of fixations cannot be mapped and are discarded. AST alignment yields 1 565 scan paths, 914 reading and 651 writing, each as an ordered sequence of fixations on AST leaf tokens (Zhang et al., 22 Aug 2025).

The method builds semantic salience priors by Bayesian Beta inference and sequential gaze-transition tables by bigram and trigram counting over semantic labels. For class $32$4, the posterior mean

$32$5

is computed from $32$6 and $32$7. Higher-order transition tables estimate $32$8 and $32$9, with n-grams of count less than 5 pruned. A pseudo-mask and pseudo scan path are then generated, AST-level weights are projected equally to all tokenizer shards, and the final per-token weight is

$C=5$00

These weights modify supervised fine-tuning without altering Transformer layers, attention heads, or embedding matrices:

$C=5$01

The framework also adds a DPO term, with total objective

$C=5$02

Training is run on StarCoder (1 B), Llama-3.2 (1 B), and DeepSeek-Coder (1.3 B) on a single NVIDIA L40S, with learning rate $C=5$03, batch size $C=5$04, epochs $C=5$05, random seed $C=5$06, DPO $C=5$07, and $C=5$08 (Zhang et al., 22 Aug 2025).

The reported task suite is CodeXGlue-based code completion, Java→C# translation, and Java→natural-language summarization. Quantitative gains are large. For StarCoder, completion CodeBLEU rises from 13.90 to 51.98, translation CodeBLEU from 52.07 to 86.42, and summarization METEOR from 29.06 to 33.41. For Llama-3.2, completion H-Exact rises from 50.85 to 77.96, translation H-Exact from 53.25 to 61.02, and summarization BERTScore from 33.41 to 49.49. For DeepSeek, completion H-Exact rises from 49.40 to 78.91, translation H-Exact from 24.13 to 65.66, and summarization BERTScore from 28.64 to 51.56 (Zhang et al., 22 Aug 2025). The ablation study attributes the largest single drop to removal of the frequency bonus: for Llama-3.2, completion H-Exact falls from 77.96 to 56.02 when that term is removed. The method does not need eye-tracking data during inference.

7. Cross-cutting distinctions, limitations, and misconceptions

The shared name obscures major methodological differences. Some EyeMulator variants are simulators in the strict sense: they generate eye-movement velocities, gaze trajectories, or synthetic eye images under specified camera and optical parameters (Fuhl et al., 2018, Fuhl et al., 2018, Lin et al., 20 Mar 2025). Others are inference systems that recover latent state from ocular measurements, such as emotive expression from IR periocular imagery in VR (Hickson et al., 2017). Still others are optical testbeds that simulate how an eye or an implanted lens forms images (Akyazi et al., 2023, Zolfaghari et al., 22 Dec 2025), or training procedures that emulate human attention statistics in a language-model loss (Zhang et al., 22 Aug 2025).

Another misconception is that all EyeMulator work is about eye tracking. The 2025 CodeLLM paper uses eye-tracking data only to derive token weights and gaze-transition statistics; the resulting model is a CodeLLM fine-tuning method, not an eye tracker (Zhang et al., 22 Aug 2025). Conversely, the 2025 synthetic-data pipeline is about predicting gaze-estimation performance under hardware changes, not about simulating human scan paths (Lin et al., 20 Mar 2025). The ophthalmic papers are closer to vision simulators and IOL characterization platforms than to gaze-estimation systems (Akyazi et al., 2023, Zolfaghari et al., 22 Dec 2025).

The limitations are likewise domain-specific. The VR expression system is constrained by upper-face labels, headset fit variation, and the select subset of expressions inferable from IR eye images alone (Hickson et al., 2017). The eye-movement simulators depend on parameter ranges and on fidelity of the Gamma, sigmoid, sampling, and noise models to the intended deployment regime (Fuhl et al., 2018, Fuhl et al., 2018). The synthetic eye-tracker prototyping pipeline is limited by identity diversity, $C=5$09 gaze range, monocular rendering, and 850 nm wavelength (Lin et al., 20 Mar 2025). The ophthalmic simulators face coherence artifacts, average-eye assumptions, and the need for patient-specific calibration (Akyazi et al., 2023, Zolfaghari et al., 22 Dec 2025). The CodeLLM method is bounded by student-Java data, only-Java eye-tracking artifacts, and the assumption that more fixations imply higher semantic importance (Zhang et al., 22 Aug 2025).

A plausible implication is that “EyeMulator” has become a productive label for emulation systems in which ocular measurements, eye-centered optics, or human visual-attention statistics are treated as computational primitives. The literature, however, does not define a single standard architecture under that name.