TimeGazer: Predictive Gaze Stabilization in AR

• TimeGazer is a temporal modeling system that reformulates gaze stabilization as a sequence-to-sequence regression problem, predicting idealized fixation trajectories from historical gaze data.
• It employs a three-stage architecture with a TimesNet backbone, synthetic data generation, and learnable convex fusion to enhance spatial concentration and accuracy.
• User studies indicate significant improvements in fixation accuracy, reduced task duration, and enhanced interaction performance compared to native AR gaze tracking.

TimeGazer is a system for temporal modeling of predictive gaze stabilization in task-driven augmented reality interaction. It reformulates gaze stabilization as a sequence-to-sequence temporal regression problem in which historical gaze dynamics from the target-searching phase are used to predict idealized fixation trajectories for the target-fixation phase. The reported system combines a TimesNet-based temporal backbone with synthetic data generation and blending that produces spatially concentrated, target-centered fixation references aligned with task objectives, and it is trained and evaluated on a hybrid dataset of real and augmented gaze sequences collected via Microsoft HoloLens 2 from 54 participants across multiple prediction horizons. The reported user study shows statistically significant improvements in interaction accuracy and completion time relative to native headset gaze tracking (Xia et al., 2 Oct 2025).

1. Task formulation and operational setting

TimeGazer addresses a specific instability regime in immersive AR: fixation sequences captured in active gaze tasks often exhibit irregular dispersion and systematic deviations from target locations. The reported causes are the combined effects of human oculomotor physiology, insufficient AR headset tracking and calibration accuracy, and environmental disturbances, with the consequence that interaction performance and visual engagement are undermined (Xia et al., 2 Oct 2025).

The system observes an egocentric, task-driven gaze sequence during the target-searching, or saccade, phase,

$$X_{1:T}=\{x_1,x_2,\ldots,x_T\}, \qquad x_t\in\mathbb{R}^d,$$

and predicts an idealized fixation sequence over a horizon $\tau$,

$$Y_{T+1:T+\tau}=\{y_{T+1},\ldots,y_{T+\tau}\}, \qquad y_k\in\mathbb{R}^d.$$

The mapping is parameterized as

$\hat{Y}_{T+1:T+\tau}=f_\theta(X_{1:T}).$

The stated objective is that predicted points should be closer to the true target and exhibit tighter spatial cohesion than the raw fixation observations (Xia et al., 2 Oct 2025).

This formulation places TimeGazer in a predictive correction setting rather than a descriptive scanpath-modeling setting. A common misunderstanding is to equate it with generic gaze prediction; the reported task is narrower and more intervention-oriented, because the model is trained to output idealized fixation trajectories specifically for the target-fixation phase of AR interaction (Xia et al., 2 Oct 2025).

2. Architectural design

TimeGazer is a 3-stage seq2seq regressor built on TimesNet, with batch size $B$, history length $T$, number of gaze features $C$, and embedding dimension $d=16$ (Xia et al., 2 Oct 2025).

The first stage is an embedding module. Each feature channel is standardized along time:

$$\tilde{x}_{b,t,c}=\frac{x_{b,t,c}-\mu_{b,c}}{\sigma_{b,c}}, \qquad \mu_{b,c}=\frac{1}{T}\sum_{t=1}^{T}x_{b,t,c}, \qquad \sigma^2_{b,c}=\frac{1}{T}\sum_{t=1}^{T}(x_{b,t,c}-\mu_{b,c})^2.$$

A token embedding is then obtained with a 1D convolution of kernel size $3$,

$\tau$0

This is combined with sinusoidal positional encoding

$\tau$1

and a bias-free timestamp linear embedding $\tau$2. The fused representation is

$\tau$3

A Predict-Linear layer then expands the temporal length from $\tau$4 to $\tau$5:

$\tau$6

The second stage is the TimesNet backbone:

$\tau$7

The summary specifies that TimesNet is a stack of TimesBlocks with Fourier-domain periodicity extraction and multi-scale convolutions (Xia et al., 2 Oct 2025). This is significant because the target signal is temporal and potentially quasi-periodic, yet corrupted by search-phase variability and device noise.

The third stage is a project layer with two parallel projections. One branch uses Multi-Head Attention with $\tau$8 heads,

$\tau$9

and the other uses a linear projection,

$$Y_{T+1:T+\tau}=\{y_{T+1},\ldots,y_{T+\tau}\}, \qquad y_k\in\mathbb{R}^d.$$0

The final prediction is a learned convex fusion,

$$Y_{T+1:T+\tau}=\{y_{T+1},\ldots,y_{T+\tau}\}, \qquad y_k\in\mathbb{R}^d.$$1

with $$Y_{T+1:T+\tau}=\{y_{T+1},\ldots,y_{T+\tau}\}, \qquad y_k\in\mathbb{R}^d.$$2 learnable (Xia et al., 2 Oct 2025).

3. Learning objective, synthetic references, and optimization

Training is driven by a combined sequence-to-sequence regression loss between the prediction $$Y_{T+1:T+\tau}=\{y_{T+1},\ldots,y_{T+\tau}\}, \qquad y_k\in\mathbb{R}^d.$$3 and a reference sequence $$Y_{T+1:T+\tau}=\{y_{T+1},\ldots,y_{T+\tau}\}, \qquad y_k\in\mathbb{R}^d.$$4 derived from real and synthetic augmentation. For a predicted fixation segment $$Y_{T+1:T+\tau}=\{y_{T+1},\ldots,y_{T+\tau}\}, \qquad y_k\in\mathbb{R}^d.$$5 and ground truth $$Y_{T+1:T+\tau}=\{y_{T+1},\ldots,y_{T+\tau}\}, \qquad y_k\in\mathbb{R}^d.$$6, the combined per-step loss is

$$Y_{T+1:T+\tau}=\{y_{T+1},\ldots,y_{T+\tau}\}, \qquad y_k\in\mathbb{R}^d.$$7

The three terms correspond respectively to MSE, center-distance, and dispersion consistency, with hyperparameters $$Y_{T+1:T+\tau}=\{y_{T+1},\ldots,y_{T+\tau}\}, \qquad y_k\in\mathbb{R}^d.$$8 and $$Y_{T+1:T+\tau}=\{y_{T+1},\ldots,y_{T+\tau}\}, \qquad y_k\in\mathbb{R}^d.$$9. The full objective adds a velocity-mismatch penalty and $\hat{Y}_{T+1:T+\tau}=f_\theta(X_{1:T}).$0 weight decay:

$\hat{Y}_{T+1:T+\tau}=f_\theta(X_{1:T}).$1

where $\hat{Y}_{T+1:T+\tau}=f_\theta(X_{1:T}).$2 is MSE of consecutive displacement magnitudes (Xia et al., 2 Oct 2025).

A distinctive component of TimeGazer is its synthetic data generation and blending strategy. After thresholded fixation extraction, real fixations are said to retain bias and dispersion. To produce synthetic “ideal” subsequences, the method contracts the fixation-phase portion of a raw subsequence toward the true target $\hat{Y}_{T+1:T+\tau}=f_\theta(X_{1:T}).$3. Given a raw subsequence

$\hat{Y}_{T+1:T+\tau}=f_\theta(X_{1:T}).$4

and boundary index $\hat{Y}_{T+1:T+\tau}=f_\theta(X_{1:T}).$5 marking the end of the search phase, the synthetic version is

$\hat{Y}_{T+1:T+\tau}=f_\theta(X_{1:T}).$6

with

$\hat{Y}_{T+1:T+\tau}=f_\theta(X_{1:T}).$7

and

$\hat{Y}_{T+1:T+\tau}=f_\theta(X_{1:T}).$8

Real and synthetic subsequences are blended during training, and the exact blend ratio was chosen to maximize validation CI and AI metrics (Xia et al., 2 Oct 2025).

Optimization uses sliding-window generation with window $\hat{Y}_{T+1:T+\tau}=f_\theta(X_{1:T}).$9 and stride $B$0, explicitly to avoid pure teacher forcing. The optimizer is Adam with initial learning rate $B$1, cosine annealing to $B$2 over $B$3 epochs, and early stopping with patience $B$4. The batch size is $B$5, the model history length is $B$6 for sliding-window training, and the prediction horizon $B$7 varies by experiment (Xia et al., 2 Oct 2025).

4. Data, metrics, and empirical performance

The reported dataset comprises 54 participants, with demographic counts of 29 male and 25 female participants aged 19–25. The collection protocol produced $B$8 rounds $B$9 raw trials, reduced after cleaning to 3771 valid trials. Each trial contains approximately 300 gaze points at 60 Hz, and the recorded features are gaze position, linear and angular velocity, head pose, timestamps, and target coordinates. The train/validation/test split is 80\%/10\%/10\% of trials, stratified by participant. For each real fixation subsequence, one synthetic variant is generated via the contraction equations above. Prediction horizons considered in theory are $T$0, while the user study uses $T$1, corresponding to approximately 1 second (Xia et al., 2 Oct 2025).

Three primary offline metrics are defined for each fixation sequence of $T$2 points. Concentration Improvement (CI) is

$T$3

with higher values better. Accuracy Improvement (AI) is

$T$4

again with higher values better. Average Distance (AD) is

$T$5

with lower values better (Xia et al., 2 Oct 2025).

For theoretical offline evaluation, the summary reports an example at $T$6 and batch size $T$7: $T$8, $T$9, $C$0, and runtime per round of approximately $C$1 s. Longer horizons are reported to yield larger AI and CI at the cost of latency (Xia et al., 2 Oct 2025).

The live user study includes $C$2 participants and a gaze-selection task over 8 sequential spheres with radii 0.10, 0.06, and 0.04 m. Relative to native HoloLens 2 gaze tracking, TimeGazer improves fixation-completion rate from $C$3 to $C$4 with $C$5 and Cohen’s $C$6; reduces task duration from $C$7 s to $C$8 s with $C$9 and $d=16$0; reduces gaze interruptions from $d=16$1 to $d=16$2 with $d=16$3 and $d=16$4; and increases fixation-to-threshold ratio for failures from $d=16$5 to $d=16$6 with $d=16$7 and $d=16$8 (Xia et al., 2 Oct 2025).

Additional MRTK scenarios show comparable patterns. In Target Selection, average task duration decreases from $d=16$9 s to $$\tilde{x}_{b,t,c}=\frac{x_{b,t,c}-\mu_{b,c}}{\sigma_{b,c}}, \qquad \mu_{b,c}=\frac{1}{T}\sum_{t=1}^{T}x_{b,t,c}, \qquad \sigma^2_{b,c}=\frac{1}{T}\sum_{t=1}^{T}(x_{b,t,c}-\mu_{b,c})^2.$$0 s with $$\tilde{x}_{b,t,c}=\frac{x_{b,t,c}-\mu_{b,c}}{\sigma_{b,c}}, \qquad \mu_{b,c}=\frac{1}{T}\sum_{t=1}^{T}x_{b,t,c}, \qquad \sigma^2_{b,c}=\frac{1}{T}\sum_{t=1}^{T}(x_{b,t,c}-\mu_{b,c})^2.$$1 and $$\tilde{x}_{b,t,c}=\frac{x_{b,t,c}-\mu_{b,c}}{\sigma_{b,c}}, \qquad \mu_{b,c}=\frac{1}{T}\sum_{t=1}^{T}x_{b,t,c}, \qquad \sigma^2_{b,c}=\frac{1}{T}\sum_{t=1}^{T}(x_{b,t,c}-\mu_{b,c})^2.$$2, and gaze interruptions decrease from $$\tilde{x}_{b,t,c}=\frac{x_{b,t,c}-\mu_{b,c}}{\sigma_{b,c}}, \qquad \mu_{b,c}=\frac{1}{T}\sum_{t=1}^{T}x_{b,t,c}, \qquad \sigma^2_{b,c}=\frac{1}{T}\sum_{t=1}^{T}(x_{b,t,c}-\mu_{b,c})^2.$$3 to $$\tilde{x}_{b,t,c}=\frac{x_{b,t,c}-\mu_{b,c}}{\sigma_{b,c}}, \qquad \mu_{b,c}=\frac{1}{T}\sum_{t=1}^{T}x_{b,t,c}, \qquad \sigma^2_{b,c}=\frac{1}{T}\sum_{t=1}^{T}(x_{b,t,c}-\mu_{b,c})^2.$$4 with $$\tilde{x}_{b,t,c}=\frac{x_{b,t,c}-\mu_{b,c}}{\sigma_{b,c}}, \qquad \mu_{b,c}=\frac{1}{T}\sum_{t=1}^{T}x_{b,t,c}, \qquad \sigma^2_{b,c}=\frac{1}{T}\sum_{t=1}^{T}(x_{b,t,c}-\mu_{b,c})^2.$$5 and $$\tilde{x}_{b,t,c}=\frac{x_{b,t,c}-\mu_{b,c}}{\sigma_{b,c}}, \qquad \mu_{b,c}=\frac{1}{T}\sum_{t=1}^{T}x_{b,t,c}, \qquad \sigma^2_{b,c}=\frac{1}{T}\sum_{t=1}^{T}(x_{b,t,c}-\mu_{b,c})^2.$$6. In Navigation, average task duration decreases from $$\tilde{x}_{b,t,c}=\frac{x_{b,t,c}-\mu_{b,c}}{\sigma_{b,c}}, \qquad \mu_{b,c}=\frac{1}{T}\sum_{t=1}^{T}x_{b,t,c}, \qquad \sigma^2_{b,c}=\frac{1}{T}\sum_{t=1}^{T}(x_{b,t,c}-\mu_{b,c})^2.$$7 s to $$\tilde{x}_{b,t,c}=\frac{x_{b,t,c}-\mu_{b,c}}{\sigma_{b,c}}, \qquad \mu_{b,c}=\frac{1}{T}\sum_{t=1}^{T}x_{b,t,c}, \qquad \sigma^2_{b,c}=\frac{1}{T}\sum_{t=1}^{T}(x_{b,t,c}-\mu_{b,c})^2.$$8 s with $$\tilde{x}_{b,t,c}=\frac{x_{b,t,c}-\mu_{b,c}}{\sigma_{b,c}}, \qquad \mu_{b,c}=\frac{1}{T}\sum_{t=1}^{T}x_{b,t,c}, \qquad \sigma^2_{b,c}=\frac{1}{T}\sum_{t=1}^{T}(x_{b,t,c}-\mu_{b,c})^2.$$9 and $3$0 (Xia et al., 2 Oct 2025).

Subjective ratings obtained with Wilcoxon signed-rank tests on 5-point Likert scales also favor TimeGazer. Stability is reported as $3$1 versus $3$2 for native HoloLens 2 gaze tracking, with $3$3 and effect size $3$4; sensitivity is $3$5 versus $3$6, with $3$7 and effect size $3$8 (Xia et al., 2 Oct 2025).

5. Position within spatio-temporal modeling research

TimeGazer belongs to a broader class of models that couple temporal structure with spatial prediction, but its target problem is distinct from both conventional scanpath prediction and image metadata inference. A useful comparison is TPP-Gaze, which models a scanpath on image $3$9 as a sequence of fixations

$\tau$00

casts gaze as a marked Neural Temporal Point Process, and learns the joint conditional density of fixation location and inter-arrival time via a scene encoder, a history encoder, and mixture-density decoders. TPP-Gaze uses a pre-trained DenseNet201 with CoordConv for scene encoding, GRU, LSTM, or Transformer modules for history encoding, and a Log-Gaussian Mixture Model for durations together with a 2D Gaussian Mixture Model for positions; its training objective is negative log-likelihood over scanpaths, and it is evaluated on COCO-FreeView, MIT1003, OSIE, NUSEF, and FiFa (D'Amelio et al., 2024).

The contrast with TimeGazer is methodological and operational. TPP-Gaze learns a probabilistic model of scanpath dynamics in free-viewing and related settings, whereas TimeGazer learns a seq2seq regression map from the search phase to an idealized fixation sequence for interactive AR stabilization (D'Amelio et al., 2024). This suggests that the two systems address different points in the perception–interaction pipeline: one models gaze behavior, while the other predicts a corrected fixation trajectory for downstream control.

A second comparison arises from GT-Loc, which jointly predicts the capture time and geo-location of an image by aligning image, time, and location embeddings in a shared $\tau$01 space. GT-Loc uses separate encoders for images, GPS coordinates, and timestamps, introduces a temporal metric-learning objective with soft targets over a cyclical toroidal surface, and performs cosine-similarity–based retrieval for month-hour and GPS prediction (Shatwell et al., 14 Jul 2025). Although GT-Loc is not a gaze model, it exemplifies the same high-level principle that “when” and “where” can be jointly structured. TimeGazer applies that broader spatio-temporal intuition to predictive gaze stabilization rather than metadata recovery or geo-localization (Xia et al., 2 Oct 2025).

6. Interpretation, limitations, and future directions

The reported discussion emphasizes two immediate effects. First, by predicting where the eye will stabilize, TimeGazer reduces within-fixation dispersion and systematic offset, reflected by increased CI and AI together with decreased AD. Second, its real-time inference time of less than 60 ms allows stabilized gaze feedback to be presented continuously, which is associated with reduced gaze interruptions and lower overall task time (Xia et al., 2 Oct 2025).

The system is also described as generalizing across users and tasks without scene semantics or heavy multimodal inputs. A plausible implication is that its utility depends less on explicit environmental understanding than on temporal regularities in search-to-fixation dynamics. In the reported discussion, this leads directly to prospective integration into gaze-based selection, dwell-control, and foveation pipelines (Xia et al., 2 Oct 2025).

Several boundary conditions are explicit in the reported account. TimeGazer is trained on a hybrid dataset of real and augmented sequences, and its fixation references are partly synthetic by construction. The paper frames this as a remedy for residual bias and dispersion in real fixations, but it also implies that the target of learning is an idealized fixation process rather than unmodified human behavior (Xia et al., 2 Oct 2025).

The stated future directions are variable-length history modeling, for example with hierarchical Transformers; uncertainty estimation; distance-aware augmentation; and extension to reading, mid-air typing, and collaborative AR/VR tasks. These directions are consistent with the system’s present formulation as a temporal predictor of stabilized fixation trajectories rather than a full generative account of gaze behavior (Xia et al., 2 Oct 2025).