PriorRG: Patient-Specific Chest X-ray Report Generation

• The paper introduces PriorRG, a framework that uses patient-specific clinical context and prior images to generate accurate chest X-ray reports.
• The two-stage architecture employs prior-guided contrastive pre-training and coarse-to-fine decoding with RAD-DINO, CXR-BERT, and DistilGPT2.
• Empirical results on MIMIC datasets show significant improvements in BLEU, METEOR, and clinical efficacy over single-image baselines.

PriorRG is a chest X-ray report generation framework that incorporates patient-specific prior knowledge—specifically clinical context and the most recent prior image—into both representation learning and report decoding. It is designed to emulate routine radiology workflow, in which radiologists interpret the current study in light of the exam indication, medical history, and temporal comparison with prior imaging. The framework was introduced in “PriorRG: Prior-Guided Contrastive Pre-training and Coarse-to-Fine Decoding for Chest X-ray Report Generation” (Liu et al., 7 Aug 2025). In this formulation, the task is to generate the Findings section from the current image $x_i^{cur}$, the most recent prior image $x_i^{pri}$ when available, the indication $z_i$, and the medical history $h_i$, with the explicit goal of producing reports that are context-aware and disease progression-oriented (Liu et al., 7 Aug 2025).

1. Clinical motivation and problem formulation

PriorRG is motivated by a limitation of single-image report generation systems: they neglect the patient-specific prior knowledge that radiologists routinely use in practice. The framework distinguishes two such priors. The first is clinical context, consisting of the indication and history, which provides the diagnostic question and helps determine which findings are salient. The second is the most recent prior image, which enables temporal reasoning about whether abnormalities are new, stable, improved, or worsened (Liu et al., 7 Aug 2025).

This design targets several failure modes identified in earlier report generation work. Methods using only the current image do not capture diagnostic intent or longitudinal progression. Methods using clinical context only improve personalization but do not model temporal change. Methods using prior images only can model progression but ignore the exam-specific diagnostic focus conveyed by indication and history. PriorRG addresses these gaps by jointly modeling temporal visual change and clinical context, with the stated aim of improving both cross-modal alignment and final report generation (Liu et al., 7 Aug 2025).

At the task level, the output is a generated report $\hat y_i$ conditioned on multimodal patient-specific inputs. The paper frames this as a clinically grounded alternative to generic medical image captioning: the target is not merely lexical similarity to a reference sentence, but a report that aligns with diagnostic focus, captures disease progression, and reduces clinically incorrect or hallucinated statements (Liu et al., 7 Aug 2025).

2. Two-stage architecture and information flow

PriorRG is organized as a two-stage pipeline. Stage 1 performs prior-guided contrastive pre-training, using clinical context to guide spatiotemporal feature extraction and align image-side representations with report semantics. Stage 2 performs prior-aware coarse-to-fine decoding, progressively integrating patient-specific prior knowledge with hierarchical visual representations to generate the report (Liu et al., 7 Aug 2025).

The architecture contains the following main components:

• Image encoder: RAD-DINO, used for current and prior images and frozen throughout training.
• Text encoder: CXR-BERT, used to encode indication, history, and reports.
• Spatiotemporal Fusion Network (STF): a ViT-style module with 3 blocks for current–prior image comparison.
• Perceiver modules: used to condense clinical context, derive clinically informed spatiotemporal features, and refine hierarchical visual semantics.
• Attention-enhanced Layer Fusion network (ALF): fuses vision encoder hidden states with CBAM-based attention and Conv2D-based channel fusion.
• Report generator: DistilGPT2, fine-tuned for autoregressive report generation (Liu et al., 7 Aug 2025).

The information flow is sequential. Current and prior images are encoded visually, clinical context is encoded textually, temporal image comparison produces a spatiotemporal representation, and condensed clinical context is then used to guide extraction of the most relevant temporal information. In the decoding stage, this coarse patient-specific prior is combined with hierarchical visual features to produce a refined conditioning signal for the LLM (Liu et al., 7 Aug 2025).

A central organizing principle is the distinction between coarse and fine information. Coarse information consists of condensed clinical context and clinically informed spatiotemporal features derived from the last hidden states. Fine information consists of hierarchical visual semantics aggregated from multiple encoder layers. The decoder therefore receives a progression from high-level patient-specific priors to fine-grained visual detail, rather than a single undifferentiated visual token sequence (Liu et al., 7 Aug 2025).

3. Prior-guided contrastive pre-training

Stage 1 is a multimodal alignment stage in which clinical context explicitly guides spatiotemporal representation learning. Visual features are extracted from the current and prior chest X-rays by RAD-DINO, with a learnable view-position embedding added to account for view-dependent variation. Textual features are extracted from indication, history, and reports using CXR-BERT, with prepended type tokens [INDICATION], [HISTORY], and [FINDINGS] (Liu et al., 7 Aug 2025).

Temporal comparison between current and prior images is performed by the STF module. Its block-level update is

$$\boldsymbol{V}^{st}_{ca} = \text{LN}(\boldsymbol{V}^{cur}+\text{CA}(\text{LN}(\boldsymbol{V}^{cur}), \text{LN}(\boldsymbol{V}^{pri})))$$

$$\boldsymbol{V}^{st} = \text{LN}(\boldsymbol{V}^{st}_{ca} + \text{FFN}(\boldsymbol{V}^{st}_{ca}))$$

where $\boldsymbol{V}^{cur}$ and $\boldsymbol{V}^{pri}$ denote current- and prior-image features. If the prior image is unavailable, the model uses $\boldsymbol{V}^{st}=\boldsymbol{V}^{cur}$ directly (Liu et al., 7 Aug 2025).

Clinical context is not merely concatenated after visual encoding. Instead, it is first condensed through a Perceiver:

$x_i^{pri}$0

and then used to guide extraction of clinically informed spatiotemporal features:

$x_i^{pri}$1

This mechanism is the core of the “prior-guided” formulation: temporal visual features are filtered through patient-specific diagnostic context, so the representation is intended to align with the intrinsic spatiotemporal semantics of radiology reports (Liu et al., 7 Aug 2025).

Cross-modal alignment is performed at the instance level. The image-to-report similarity distribution is

$x_i^{pri}$2

with a symmetric report-to-image distribution $x_i^{pri}$3. Because multiple image-report pairs can correspond to the same visit, the ground-truth label matrix allows multiple positives:

$x_i^{pri}$4

The Stage-1 loss is the bidirectional alignment loss

$x_i^{pri}$5

which is the full Stage-1 objective (Liu et al., 7 Aug 2025).

4. Prior-aware coarse-to-fine decoding

Stage 2 generates the report by progressively integrating patient-specific priors with visual evidence. The decoder-side representation begins with coarse priors: the condensed clinical context $x_i^{pri}$6 and the clinically informed spatiotemporal feature $x_i^{pri}$7. These are then used to refine hierarchical visual features derived from the vision encoder hidden states (Liu et al., 7 Aug 2025).

The hierarchical feature stream is produced by the Attention-enhanced Layer Fusion network. ALF takes hidden states from multiple RAD-DINO layers, applies CBAM channel and spatial attention, uses a Conv2D-based channel fusion projector, and outputs a hierarchical visual representation

$x_i^{pri}$8

The paper describes ALF in prose and attributes its purpose to preserving low-level lesion morphology and higher-level semantics that may be lost when only the final hidden state is used (Liu et al., 7 Aug 2025).

The coarse-to-fine refinement step is

$x_i^{pri}$9

This defines “coarse-to-fine” in PriorRG: coarse patient-specific priors first establish diagnostic focus and temporal context, and then those priors attend into fine-grained hierarchical visual semantics. The resulting conditioning sequence is the concatenation

$z_i$0

which is fed to DistilGPT2 for report generation (Liu et al., 7 Aug 2025).

The Stage-2 objective is the autoregressive conditional language modeling loss

$z_i$1

with no additional auxiliary decoding loss reported. The paper does not describe separate sentence-planning, diagnostic-focus, or progression-specific decoder heads; the coarse-to-fine behavior is implemented by progressive feature construction and concatenated conditioning rather than by multiple decoders or multi-pass generation (Liu et al., 7 Aug 2025).

5. Data, optimization, and implementation

PriorRG is evaluated primarily on MIMIC-CXR and MIMIC-ABN, with additional cross-dataset experiments on IU X-ray. For MIMIC-CXR, the training split contains 239,998 images, 239,998 samples, and 150,957 reports; 60.5% of training samples have a prior image, 66.4% have indication, and 30.6% have history. For MIMIC-ABN, the training split contains 69,641 images, 69,641 samples, and 34,763 reports; 52.7% have a prior image, 64.6% have indication, and 34.9% have history (Liu et al., 7 Aug 2025).

Data are organized by study id, and the most recent prior image is retrieved when available. The framework is designed to tolerate missing priors. This is operationalized through unified text encoding with special tokens and the fallback $z_i$2 when no prior image exists (Liu et al., 7 Aug 2025).

The principal implementation settings reported in the paper are summarized below.

Component	Setting
Unified feature dimension	$z_i$3
Perceiver latents	$z_i$4
STF blocks	3
Maximum generation length	$z_i$5
Beam size	3

Optimization uses AdamW with ReduceLROnPlateau, scheduler patience 5, and early stopping patience 15. RAD-DINO is frozen throughout training, while CXR-BERT, DistilGPT2, the projection heads, ALF, and STF are fine-tuned. For MIMIC-CXR, Stage 1 uses 30 epochs, batch size 32, and learning rate $z_i$6. Stage 2 uses 30 epochs, batch size 16, learning rate $z_i$7 for the report generator and $z_i$8 for the remaining components. MIMIC-ABN fine-tuning is initialized from MIMIC-CXR pretrained weights and uses 30 epochs, batch size 16, and learning rate $z_i$9. Training is performed on a single NVIDIA V100 GPU with approximately 26 GB memory usage (Liu et al., 7 Aug 2025).

6. Empirical performance, ablations, and limitations

On MIMIC-CXR, PriorRG reports B-1 0.412, B-2 0.290, B-3 0.220, B-4 0.175, METEOR 0.189, ROUGE-L 0.324, Precision 0.541, Recall 0.485, and F1 0.511. The paper reports gains over the best baseline of +3.6% BLEU-4, +1.3% METEOR, +1.5% ROUGE-L, and +3.8% F1. On MIMIC-ABN, PriorRG reports B-1 0.326, B-2 0.201, B-3 0.139, B-4 0.102, METEOR 0.140, ROUGE-L 0.242, Precision 0.467, Recall 0.476, and F1 0.471, with reported gains of +5.9% BLEU-1, +4.4% BLEU-2, +3.5% BLEU-3, +2.9% BLEU-4, +2.6% METEOR, +1.1% ROUGE-L, and +1.1% F1 (Liu et al., 7 Aug 2025).

The paper also reports improvements in clinical efficacy. Relative to SEI, PriorRG improves recall on 12 of 14 CheXpert observations and F1 on 13 of 14. Example F1 changes reported in Table 2 include Pneumonia $h_i$0, Pneumothorax $h_i$1, Fracture $h_i$2, Support Devices $h_i$3, and No Finding $h_i$4. The micro-average F1 increases from 0.460 to 0.511 (Liu et al., 7 Aug 2025).

Stage 1 is supported by retrieval results on MIMIC-5x200. PriorRG reports Stu-P@1 0.402, Stu-P@3 0.334, Stu-P@5 0.251, Cat-P@1 0.640, Cat-P@3 0.596, and Cat-P@5 0.544. This suggests that the prior-guided contrastive stage improves both same-study and disease-category retrieval. The paper’s ablation results further indicate that clinical context helps category-level semantic alignment, whereas prior images especially help study-level retrieval (Liu et al., 7 Aug 2025).

Ablation experiments isolate the contribution of both stages and of the patient-specific priors. Removing Stage 1 reduces B-4 from 0.175 to 0.165 and F1 from 0.511 to 0.459. Omitting Stage 2 yields B-4 0.108 and F1 0.472. Excluding patient-specific prior knowledge yields B-4 0.102 and F1 0.496. Removing hierarchical visual fusion reduces B-4 from 0.175 to 0.171 and F1 from 0.511 to 0.499. The progressive fusion analysis reports LastOnly at B-4 0.168 and F1 0.506, Fine2coarse at B-4 0.172 and F1 0.514, and PriorRG at B-4 0.175 and F1 0.511; this indicates that retaining intermediate priors and using the proposed coarse-to-fine order favors overall linguistic quality, while the reverse order slightly raises F1 (Liu et al., 7 Aug 2025).

Cross-dataset experiments on IU X-ray show zero-shot performance of BLEU-4 0.178 and METEOR 0.211, and supervised fine-tuning performance of BLEU-4 0.196 and METEOR 0.216. In comparison with concurrent MLRG on MIMIC-CXR, PriorRG reports B-4 0.175 versus 0.158, F1 0.511 versus 0.505, F1-RadGraph 0.296 versus 0.293, time per sample 0.1243s versus 0.2703s, and GPU memory 7.68GB versus 12.14GB (Liu et al., 7 Aug 2025).

The primary limitation explicitly identified in the paper is imperfect modeling of fine-grained medical attributes. The failure analysis notes overstatement errors, such as simplifying “mildly enlarged cardiac silhouette” to “enlarged.” The authors attribute this to insufficient modeling of severity and propose future work based on attributed abnormality graphs. They also state an intention to explore organ-aware diagnosis frameworks to improve interpretability. A plausible implication is that PriorRG’s gains arise from improved patient-specific conditioning, while the remaining error budget is concentrated in attribute precision rather than in gross omission of context or temporal information (Liu et al., 7 Aug 2025).