RadGazeIntent: Radiologist Intention from Eye Fixations

• RadGazeIntent is a transformer-based method that interprets radiologist eye fixation sequences to predict the diagnostic findings being examined.
• It combines high-resolution foveal mapping with peripheral features and spatial-temporal embeddings to capture both local and global image context.
• By modeling latent diagnostic intentions rather than simple saliency, the approach paves the way for intention-aware medical AI and enhanced human-AI collaboration.

Searching arXiv for the target paper and closely related gaze-intent work to ground the article with current references. {"query":"RadGazeIntent Interpreting Radiologist's Intention from Eye Movements in Chest X-ray Diagnosis arXiv (Pham et al., 16 Jul 2025) related radiology gaze intent chest x-ray", "max_results": 10} Fetching the most relevant records for radiology gaze-intent and adjacent gaze-sequence methods. {"query":"all:(Pham et al., 16 Jul 2025) OR ti:\"Interpreting Radiologist's Intention from Eye Movements in Chest X-ray Diagnosis\" OR ti:\"RadEyeVideo\" OR ti:\"Radiologist\" eye gaze chest x-ray intention", "max_results": 10} Searching directly for adjacent methods on radiology gaze sequence modeling and gaze-conditioned medical LVLMs. {"query":"ti:\"RadEyeVideo\" OR ti:\"RadGazeGen\" OR ti:\"Chest X-ray\" eye gaze arXiv", "max_results": 10} RadGazeIntent is a transformer-based method for fixation-level intention interpretation in chest X-ray reading. It addresses a gaze-understanding problem that is explicitly different from modeling saliency, localization, report generation, or scanpath prediction: given a radiologist’s fixation sequence, the system predicts which radiological finding the radiologist is examining at each moment, rather than merely where the radiologist looked. The method is introduced in “Interpreting Radiologist's Intention from Eye Movements in Chest X-ray Diagnosis” (Pham et al., 16 Jul 2025), which frames radiologist eye movements as behavioral signals shaped by latent diagnostic goals and operationalizes intention as a coarser cognitive state that can persist across multiple adjacent fixations.

1. Conceptual framing and scope

RadGazeIntent is defined around a specific distinction: prior gaze-assisted medical image analysis methods generally use fixations or gaze heatmaps as auxiliary supervision for disease classification, localization, report generation, or scanpath prediction, whereas RadGazeIntent aims to infer the latent diagnostic purpose behind each fixation. In the paper’s terminology, “intention” is not a pixel location, nor a single fixation event, but the radiological finding currently being examined. The authors describe this as a shift from modeling “where” experts looked, and sometimes “when,” to modeling “why” they looked there (Pham et al., 16 Jul 2025).

The paper develops three behavioral interpretations of intention. The first is a systematic sequential search, in which radiologists follow a mental checklist and search for findings one at a time. The second is an uncertainty-driven exploratory mode, in which fixations may be opportunistically related to any finding currently under consideration. The third is a hybrid pattern, in which an initial broad scan is followed by focused search for specific pathologies. These are not presented as directly observed cognitive ground truth. Rather, they are operational assumptions applied to existing gaze-report corpora.

This framing places RadGazeIntent within a broader research movement that treats gaze as more than a saliency prior. In radiology, adjacent work has also argued that ordered gaze carries information beyond static heatmaps, as in “RadEyeVideo” (Kim et al., 12 Jul 2025), while “RadGazeGen” uses radiologist gaze heatmaps as controls for diffusion-based image generation (Bhattacharya et al., 2024). Outside radiology, gaze has also been used for explicit intention recognition in action anticipation and robotics (Ozdel et al., 2024). RadGazeIntent is distinctive in that its target variable is explicitly fixation-level, multi-label diagnostic intention in chest X-ray interpretation.

2. Task formulation and representation of intention

The task is formulated as fixation-level sequence labeling. Given a fixation sequence of length $T$,

$\mathcal{F} = \{f_1, f_2, \ldots, f_T\},$

where each fixation is

$f_i = (x_i, y_i, d_i),$

the model predicts a multi-label intention vector for every fixation. The ground-truth label sequence is

$L = \{l_1, l_2, \ldots, l_T\},$

with

$l_i \in \{0,1\}^K,$

where $K$ is the number of possible intention classes, corresponding to radiological findings (Pham et al., 16 Jul 2025).

This definition is important because it fixes the prediction granularity at the level of the full scanpath rather than the image, report, or fixation heatmap. The model is explicitly described as “predicting not what findings are present but which finding a radiologist is examining at a given moment.” Duration $d_i$ appears in the task formulation, but the architectural description mainly uses fixation coordinates for image feature sampling and temporal order for sequence encoding. The authors also argue that fixations are noisy, fine-grained observations, whereas intentions are coarser latent states that often persist across multiple adjacent fixations.

The formulation is multi-label rather than single-label. That choice is tied to the paper’s exploratory and hybrid search assumptions, where a fixation may plausibly relate to more than one finding under consideration. A plausible implication is that the authors treat ambiguity as intrinsic to expert visual search rather than as annotation error alone.

Training uses multi-label binary cross-entropy over all fixation-finding pairs: $$\mathcal{L} = -\frac{1}{TK}\sum_{i=1}^{T}\sum_{k=1}^{K} \left[ l_{ik} \log(\hat{l}_{ik}) + (1-l_{ik}) \log(1-\hat{l}_{ik}) \right].$$ No auxiliary losses are described.

3. Architecture and sequence modeling

RadGazeIntent is a transformer-based sequence model operating jointly on the chest X-ray image and the fixation sequence. Its four major components are feature extraction, structural embedding, peripheral-aware causal self-attention, and an intention decoder (Pham et al., 16 Jul 2025).

The image is processed by a Feature Pyramid Network with a ResNet backbone to produce multiscale feature maps

$\{P^1, P^2, P^3, P^4\}.$

Two levels are used explicitly: $P^l=P^1 \in \mathbb{R}^{C \times H/32 \times W/32}$ for peripheral visual information, and

$\mathcal{F} = \{f_1, f_2, \ldots, f_T\},$0

for foveal visual information. Each fixation $\mathcal{F} = \{f_1, f_2, \ldots, f_T\},$1 is mapped to block $\mathcal{F} = \{f_1, f_2, \ldots, f_T\},$2 on $\mathcal{F} = \{f_1, f_2, \ldots, f_T\},$3, producing fixation-aligned image features

$\mathcal{F} = \{f_1, f_2, \ldots, f_T\},$4

where $\mathcal{F} = \{f_1, f_2, \ldots, f_T\},$5. The paper terms this step Fovea Mapping.

The model then adds explicit spatial and temporal structure. It applies 2D spatial embedding to encode image-plane location and 1D temporal embedding to encode fixation order. In parallel, the peripheral feature map is flattened into

$\mathcal{F} = \{f_1, f_2, \ldots, f_T\},$6

Peripheral tokens and fixation tokens are concatenated as

$\mathcal{F} = \{f_1, f_2, \ldots, f_T\},$7

This tokenization permits joint reasoning over global image context and fixation-conditioned local evidence.

The core transformer mechanism is Peripheral-aware Causal Self Attention. Its mask is defined over peripheral and fixation tokens as

$\mathcal{F} = \{f_1, f_2, \ldots, f_T\},$8

so that fixation tokens can attend to all peripheral tokens and to previous fixation tokens, but not future fixation tokens. The attention equation in the provided paper text is typographically corrupted, although the intended operation is masked scaled dot-product attention. The encoder stacks $\mathcal{F} = \{f_1, f_2, \ldots, f_T\},$9 such blocks and outputs

$f_i = (x_i, y_i, d_i),$0

The intention decoder converts fine-grained fixation tokens into coarser intention-like representations. It applies Pool Attention to compress the fixation sequence from $f_i = (x_i, y_i, d_i),$1 tokens into a shorter latent sequence $f_i = (x_i, y_i, d_i),$2. The paper does not give an explicit pooling equation, but states that repeated self-attention and cross-attention layers allow each original fixation representation in $f_i = (x_i, y_i, d_i),$3 to query the compressed latent set $f_i = (x_i, y_i, d_i),$4. The decoder is repeated for $f_i = (x_i, y_i, d_i),$5 layers, followed by a linear layer and sigmoid for per-fixation confidence over findings.

The reported implementation uses an FPN with a ResNet-50 backbone initialized from an MGCA-pretrained checkpoint and frozen during training. The model uses $f_i = (x_i, y_i, d_i),$6 peripheral-aware causal self-attention layers, hidden size $f_i = (x_i, y_i, d_i),$7, and $f_i = (x_i, y_i, d_i),$8 attention heads. Pool attention uses stride 2 and kernel size 5 tokens. The intention decoder uses $f_i = (x_i, y_i, d_i),$9 blocks of self-attention and cross-attention. Training runs for 4,000 iterations with AdamW, learning rate $L = \{l_1, l_2, \ldots, l_T\},$0, and batch size 32 on a single NVIDIA A6000 GPU.

4. Intention-labeled datasets and behavioral assumptions

Because intention is not directly annotated in the source corpora, the paper derives three intention-labeled subsets from two public chest X-ray eye-tracking datasets, EGD and REFLACX. All data are chest X-rays only. The three derived subsets come from a source pool of 1,079 samples from EGD and 2,483 samples from REFLACX. Evaluation uses 5-fold cross-validation rather than explicit train/validation/test splits (Pham et al., 16 Jul 2025).

The intention classes correspond to 13 findings: Atelectasis, Cardiomegaly, Consolidation, Edema, Enlarged Cardiomediastinum, Fracture, Lung Lesion, Lung Opacity, Pleural Effusion, Pleural Other, Pneumonia, Pneumothorax, and Support Devices.

Subset	Behavioral assumption	Label construction
RadSeq	Systematic sequential search	GazeSearch constraints on RadExplore with radius set to zero
RadExplore	Uncertainty-driven exploration	Sentence-level report timing plus CheXbert via “Uncertainty Filtering”
RadHybrid	Broad scan then focused search	First $L = \{l_1, l_2, \ldots, l_T\},$1 second merged with RadSeq labels

For RadExplore, the paper assumes maximal ambiguity: any fixation may potentially be relevant to any finding discussed so far. Let $L = \{l_1, l_2, \ldots, l_T\},$2 be the transcript sentence sequence, each sentence with end time $L = \{l_1, l_2, \ldots, l_T\},$3, and let $L = \{l_1, l_2, \ldots, l_T\},$4 be the fixation timestamps. CheXbert maps sentences to classes $L = \{l_1, l_2, \ldots, l_T\},$5. Labels are defined as

$L = \{l_1, l_2, \ldots, l_T\},$6

and the procedure is termed Uncertainty Filtering.

For RadSeq, the assumption is that radiologists search one finding at a time in sequence. GazeSearch constraints—radius-based filtering and time-spent constraining—are applied to RadExplore to produce beginning and end times $L = \{l_1, l_2, \ldots, l_T\},$7 and $L = \{l_1, l_2, \ldots, l_T\},$8 for each intention $L = \{l_1, l_2, \ldots, l_T\},$9: $l_i \in \{0,1\}^K,$0 The authors make one implementation change: the radius is set to zero in the radius-based filtering procedure to avoid discarding fixation points and thereby preserve temporal and spatial information.

For RadHybrid, the first phase is the scanning fixations in the first $l_i \in \{0,1\}^K,$1 seconds, with

$l_i \in \{0,1\}^K,$2

These initial fixations are merged with RadSeq labels: $l_i \in \{0,1\}^K,$3

The paper is explicit that these subsets operationalize different theories of radiologist search behavior rather than absolute ground truth intention. This suggests that the benchmark measures consistency with structured behavioral assumptions derived from gaze-report alignment, not direct access to internal diagnostic state.

5. Empirical performance and ablation evidence

Evaluation uses fixation-intention pair classification metrics: Accuracy (ACC), F1-score (F1), Precision (P), and Recall (R), reported with 5-fold cross-validation and 95% confidence intervals. Baselines include an MLP that treats fixations independently, a unidirectional LSTM over fixation tokens, a modified Karargyris et al. heatmap-based model, and an adapted ChestSearch baseline (Pham et al., 16 Jul 2025).

On RadSeq, RadGazeIntent achieves 88.85 ACC / 72.05 F1 / 74.01 P / 70.51 R on EGD and 86.92 ACC / 69.87 F1 / 72.12 P / 67.90 R on REFLACX. The strongest baseline, ChestSearch, obtains 68.20 F1 on EGD and 65.44 F1 on REFLACX, corresponding to F1 gains of about 3.85 and 4.43 points.

On RadExplore, RadGazeIntent obtains 87.95 ACC / 70.14 F1 / 72.25 P / 68.01 R on EGD and 85.40 ACC / 67.33 F1 / 70.89 P / 65.92 R on REFLACX. ChestSearch reaches 66.10 and 63.11 F1, so the gains are about 4.04 and 4.22 F1 points.

On RadHybrid, RadGazeIntent obtains 88.21 ACC / 71.11 F1 / 73.20 P / 69.88 R on EGD and 86.02 ACC / 68.44 F1 / 71.55 P / 66.78 R on REFLACX. ChestSearch scores 65.77 and 62.70 F1, so the gains are roughly 5.34 and 5.74 F1 points. The paper attributes the strongest relative improvement to the decoder’s pooling mechanism, which better separates broad exploratory fixations from focused search fixations.

Setting	RadGazeIntent F1	ChestSearch F1
RadSeq / EGD	72.05	68.20
RadSeq / REFLACX	69.87	65.44
RadExplore / EGD	70.14	66.10
RadExplore / REFLACX	67.33	63.11
RadHybrid / EGD	71.11	65.77
RadHybrid / REFLACX	68.44	62.70

The ablation study reports average F1 over EGD and REFLACX. The full model gives 71.01 on RadSeq, 68.74 on RadExplore, and 69.78 on RadHybrid. Removing Pool Attention reduces performance to 68.20 / 67.45 / 68.01. Removing 1D Temporal Embedding gives 66.33 / 64.10 / 65.44. Removing 2D Spatial Embedding yields 67.77 / 62.89 / 63.11. Removing Peripheral Feature gives 64.34 / 63.45 / 65.02. The largest degradation comes from removing Fovea Mapping and replacing it with coordinate-based layout embedding only, yielding 58.92 / 57.01 / 57.66. The paper interprets this as evidence that fixation-aligned high-resolution image evidence is essential for identifying subtle diagnostic targets.

Qualitatively, the paper overlays predicted fixation-level intention labels on chest X-rays, using green dots for correct predictions and red dots for incorrect ones at confidence threshold 0.5. It states that predictions more closely resemble the ground truth than those of Karargyris et al. or ChestSearch across RadSeq, RadExplore, and RadHybrid. It does not provide transformer attention-map visualizations.

6. Significance, limitations, and position within gaze-intent research

The paper states three concrete contributions: a new benchmark problem of fixation-level intention interpretation in radiology, three intention-labeled datasets—RadSeq, RadExplore, and RadHybrid—from existing chest X-ray eye-tracking corpora, and a transformer architecture combining foveal and peripheral image features, causal temporal modeling, and a pooling-based decoder to infer coarse latent intention states from fine-grained fixations (Pham et al., 16 Jul 2025).

Its limitations are equally explicit. Intention is abstract and not directly observed in the original eye-tracking datasets; all labels are inferred from behavioral assumptions and report timing, so annotation noise is unavoidable. Not all fixations map cleanly to a specific diagnostic objective, especially in early scanning or exploratory phases. The datasets are chest X-rays only, and the paper does not report external generalization to non-CXR modalities such as CT or MRI. It also does not report cross-dataset generalization experiments beyond evaluation on both EGD and REFLACX under the same derived task settings.

In relation to adjacent work, RadGazeIntent occupies a narrower but more explicit intent-modeling position. “RadEyeVideo” preserves radiologist scanpath order by turning gaze into a video prompt for video-capable LVLMs, improving report generation and diagnosis in some settings, but it does not infer intention as a supervised latent variable (Kim et al., 12 Jul 2025). “RadGazeGen” uses radiologist gaze as Human Visual Attention maps to control diffusion-based chest X-ray generation, emphasizing spatialized expert focus rather than fixation-level intention (Bhattacharya et al., 2024). More broadly, gaze-based intention recognition has been studied in action anticipation and human-robot interaction, including graph-based intention recognition from observer gaze (Ozdel et al., 2024), but those settings differ in modality, task structure, and label semantics.

The practical implication stated by the paper is that gaze-based medical AI can move from using radiologists merely as a source of spatial supervision toward modeling diagnostic behavior itself. A plausible implication is that such fixation-level intention modeling could support intention-aware human-AI collaboration, adaptive training tools, gaze-guided report generation, and real-time feedback systems aligned with a reader’s current diagnostic focus. The paper, however, does not claim stronger cognitive validity than its label-construction assumptions support.