GazeLT: Visual-Attention Framework for CXR

• GazeLT is a visual-attention framework that leverages temporal radiologist gaze patterns to classify both common and rare chest pathologies.
• It employs a dual teacher–student architecture with integration and disintegration attention modes across four time windows for robust feature learning.
• Empirical results on NIH-CXR-LT and MIMIC-CXR-LT datasets demonstrate significantly improved tail accuracy and overall balanced performance versus baselines.

GazeLT is a visual-attention-guided framework designed for long-tailed disease classification in chest radiographs by transferring temporally evolving radiologist visual search policies into deep models. The system leverages eye gaze data to enable automated diagnostic models to better recognize both common (head) and rare (tail) pathologies by explicitly modeling the evolution of attention during radiograph interpretation. Its central innovation is a two-branch teacher–student architecture exploiting temporally structured integration and disintegration signals, distilled to a classification student tailored for heavily imbalanced disease distributions (Bhattacharya et al., 13 Aug 2025).

1. Conceptual Foundations and Motivation

GazeLT is motivated by two observations about expert radiological search. First, radiologists' visual attention on chest X-rays is inherently temporal: inspection shifts sequentially across regions, with varying attentional scopes. Second, long-tailed disease recognition requires not only detecting prominent findings associated with frequent classes but also subtle, spatially dispersed cues of rare pathologies. The framework thus encodes both the chronology and granularity of expert inspection, formally distinguishing between 'integration' (fine-grained, focused) and 'disintegration' (global, coarse) modes of gaze, and transferring these to an automated classifier via a temporally resolved distillation process.

2. Integration–Disintegration Attention and Temporal Modeling

GazeLT operationalizes two complementary visual search modes:

• Integration Attention: Captures clustered, focused scrutiny. Fixation points within a threshold distance are grouped as main attention regions; substitute regions are formed by residual fixations. Gaussian smoothing ($\sigma=64$) forms integration heatmaps, modeling the process of concentrated diagnostic search.
• Disintegration Attention: Encodes broader, less localized scanning. Here, all fixation points are directly smoothed using a wider Gaussian ($\sigma=128$), yielding diffusely distributed attention maps corresponding to global image survey or attention shifts.

Crucially, both modes are computed over $n$ temporal windows (default $n=4$), segmenting the gaze sequence to yield temporally indexed maps $\{f_t^I\}_{t=1}^n$ (integration) and $\{f_t^D\}_{t=1}^n$ (disintegration). This temporal decomposition allows the model to learn when and how attention transitions from frequent to rare diagnostic targets, aligning different temporal windows with different clinical cues.

3. Architecture and Attention Supervision

GazeLT implements a teacher–student architecture:

• Teacher (t-HVA): Comprises two branches:
  • TW-I (Time-Windowed Integration): Uses a FocalNet transformer backbone, processing the image through four sequential sub-blocks, each predicting attention features $\mathcal{O}^I_t$ per time window. Outputs are decoded to match $f_t^I$.
  • TW-D (Time-Windowed Disintegration): Employs a GC-ViT (Global Context Vision Transformer) backbone, again with four sequential blocks, matching their outputs $\mathcal{O}^D_t$ to $f_t^D$.
• Student: A standard ResNet classifier. For each input, the student produces a feature vector $\sigma=128$0. The teacher provides branch-level features $\sigma=128$1, $\sigma=128$2, fused as $\sigma=128$3. The student is trained with a Bhattacharyya-distance distillation loss aligning $\sigma=128$4 with $\sigma=128$5 in addition to a long-tailed classification objective (LDAM loss).

The eye-gaze data (from EDG-CXR, REFLACX) is used for teacher pretraining only; student deployment requires only images.

4. Training Methodology and Loss Functions

GazeLT training proceeds in two stages:

• Teacher Pretraining:

1. Partition fixation sequence into $\sigma=128$6 windows.
2. Compute time-windowed integration ($\sigma=128$7) and disintegration ($\sigma=128$8) maps.
3. Forward the image through TW-I/TW-D to obtain $\sigma=128$9.
4. Minimize time-windowed visual attention alignment losses:

   $n$0

   $n$1

• Student Training:

1. Extract fixed teacher features for each training example.
2. Compute fused teacher representation $n$2.
3. Pass image through student; obtain $n$3, logits $n$4.
4. Optimize composite loss:

   $n$5

   with

   $n$6

   and

   $n$7

At inference, the student alone is used; attention supervision is no longer present.

5. Experimental Protocols and Quantitative Outcomes

Comprehensive evaluation is performed on two open large-scale long-tailed CXR datasets:

• NIH-CXR-LT ($n$8)
• MIMIC-CXR-LT ($n$9)

Metrics include average accuracy on balanced/imbalanced test splits, as well as group-wise head/medium/tail accuracy and multiclass diagnostics (balanced accuracy, MCC, AUC, wF1).

Key findings (balanced split):

Dataset	Method	Head	Medium	Tail	Avg
NIH-CXR-LT	GazeLT	0.404	0.411	0.417	0.410
	Decoupling-cRT	—	—	—	0.369
	RW LDAM-DRW	—	—	—	0.362
MIMIC-CXR-LT	GazeLT	0.480	0.278	0.489	0.418
	Decoupling-cRT	—	—	—	0.387

On tail classes:

• NIH-CXR-LT: tail accuracy 0.417 (best baseline: 0.308)
• MIMIC-CXR-LT: tail accuracy 0.489 (best baseline: 0.367)
• Reported improvements: +10.9% (NIH), +12.2% (MIMIC) tail accuracy, $n=4$0.

On the imbalanced split, GazeLT achieves a balanced accuracy of 0.315 (NIH) and 0.292 (MIMIC), outperforming all baselines except for a marginally lower score on MIMIC imbalanced (vs. Decoupling-cRT’s 0.296), but with stronger tail-class performance.

6. Ablations, Sensitivity, and Methodological Distinctions

Ablation studies confirm:

• Both integration and disintegration branches contribute; removal of either reduces performance.
• Time-windowed variants ('TW-I', 'TW-D') outperform static ('I', 'D') variants.
• Best aggregate performance is achieved with $n=4$1 time windows; fewer windows underresolve, too many lead to temporal redundancy and degraded tail accuracy (e.g., $n=4$2 reduces tail accuracy to 0.275).

Comparison with prior methods (e.g., GazeRadar, RadioTransformer) reveals that static gaze guidance—supervising with a single attention map—yields substantially smaller gains (e.g., GazeLT outperforms RadioTransformer by 21.7%, GazeRadar by 22.3% on NIH-CXR-LT balanced avg accuracy, both $n=4$3). Temporal windowing and decomposition into integration/disintegration are thus the principal sources of performance improvement.

Variant	NIH-CXR-LT Balanced	Imbalanced
Integration only	0.383	0.296
Disintegration	0.392	0.294
TW-I + TW-D	0.410	0.315
I + D (static)	0.363	0.295

7. Interpretation and Significance

GazeLT represents an explicit formalization of radiologist visual expertise as temporally sequenced attention, systematically transferred to deep models for imbalanced medical classification tasks. The integration–disintegration and temporal-windowing mechanism enables not just spatial but sequential imitation of expert search, which, as per empirical results, disproportionately benefits rare-class recognition—a critical need in clinical disease screening scenarios. Its two-stage pipeline—teacher attention alignment via gaze, followed by feature-level distillation and long-tailed optimization in the student classifier—proves more effective than non-temporally-resolved or static gaze supervision. The framework demonstrates that encoding expert diagnostic process at the latent representation level, rather than merely at diagnostic labels, can yield significant improvements for automated disease detection in highly imbalanced real-world tasks (Bhattacharya et al., 13 Aug 2025).