---
title: CLARIFID Radiology Report Framework
url: https://www.emergentmind.com/topics/clarifid
type: topic
---

# CLARIFID Radiology Report Framework

Searching arXiv for the CLARIFID paper and closely related clarification-oriented work to ground the article in current literature.
CLARIFID is a radiology report generation framework for chest X-ray studies that is designed to mirror the two-step workflow of radiologists: first documenting detailed **Findings**, then synthesizing a clinically accurate **Impression**. It addresses a central limitation of prior report-generation systems, namely that many optimize fluent text or whole-report similarity without directly optimizing the diagnostic correctness of the Impression section, even though the paper states that in MIMIC-CXR, **82.4% of diagnostic labels come from the impression** [2507.17234]. CLARIFID combines section-aware pretraining, Proximal Policy Optimization with **CheXbert F1 on the Impression section** as reward, reasoning-aware decoding that forces the model to complete Findings before Impression, and multi-view chest X-ray fusion through a SwinV2-based encoder [2507.17234].

## 1. Problem formulation and design rationale

CLARIFID is framed around automatic generation of full free-text chest X-ray reports from one or more input images, without using prior reports or clinical metadata. The target output contains two structured sections: **Findings**, which record detailed observations, and **Impression**, which provides the diagnostic summary that clinicians act on [2507.17234]. The framework is evaluated on **MIMIC-CXR (full)** for Findings-only comparison, on **MIMIC-CXR Impression subset (M-CXR Imp.)** for two-section report generation, and on **IU X-ray** in a cross-dataset setting where training is performed only on MIMIC-CXR and testing is performed on IU X-ray reports that contain both Findings and Impression [2507.17234].

The motivating claim is that prior methods often optimize generic NLG metrics such as BLEU, ROUGE, METEOR, and CIDEr, while not directly ensuring factual correctness in the clinically decisive part of the report. They also frequently rely on single-view images and often treat the report as an undifferentiated block of text or generate Findings only [2507.17234]. CLARIFID instead treats report generation as a structured reasoning problem in which the Findings section functions as the evidential substrate for the Impression.

This design implies a specific notion of correctness. The framework does not merely seek lexical overlap with reference reports; it seeks to improve **clinical efficacy**, measured through CheXbert-based label agreement, especially for the Impression. A plausible implication is that the framework reorients supervision from report fluency toward diagnostic faithfulness, while preserving the section structure used in actual radiology workflows.

## 2. System architecture and data flow

CLARIFID uses a multi-component architecture centered on a visual encoder, a policy network for report generation, and a value network for reinforcement learning and inference-time re-ranking [2507.17234]. The model receives up to three chest X-ray images per study, encodes them jointly, and generates a report sequence containing explicit control tokens for section structure.

Before the table, the main architectural point is that visual information from all available views is fused into a single token sequence, while text generation is constrained to respect the ordering **Findings \(\rightarrow\) Impression**. This architecture is therefore not only multimodal but also section-aware and decoding-aware.

| Component | Specification | Function |
|---|---|---|
| Vision encoder | SwinV2 | Encodes one or more chest X-ray views into visual tokens |
| Policy network | GPT-2-style decoder, 30 layers, hidden size 768, 12 attention heads | Generates the full report with cross-attention to visual tokens |
| Value network | GPT-2 + cross-attention, 6 layers, same hidden size and heads | Estimates expected reward for PPO and report-level re-ranking |

For study \(i\) with \(K_i\) images \(I_{i,j} \in \mathbb{R}^{C\times H\times W}\), the images are flattened across the batch, encoded by the shared SwinV2 backbone, regrouped per study, concatenated across views, and padded to a common maximum sequence length. The resulting padded visual tensor \(\widehat{\mathbf{S}}\) and attention mask \(\mathbf{A}\) are consumed by the decoder through cross-attention [2507.17234]:

\[
\mathcal{I} = \{ I_{i,j} \mid i\in[1,B], j \in [1,K_i] \}
\]

\[
\mathbf{F} = E(\mathcal{I}) \in \mathbb{R}^{|\mathcal{I}|\times N \times D}
\]

\[
\mathbf{S}_i \in \mathbb{R}^{K_i\cdot N \times D}
\]

\[
\widehat{\mathbf{S}} = \text{Pad}([\mathbf{S}_1, \dots, \mathbf{S}_B]) \in \mathbb{R}^{B\times M_{max}\times D}
\]

\[
\mathbf{A} = \text{Pad}([\mathbf{1}_{K_1\cdot N}, \dots, \mathbf{1}_{K_B\cdot N}]) \in \{0,1\}^{B\times M_{max}}
\]

At each decoder layer, textual hidden states first undergo self-attention, then cross-attention against the visual sequence, then feed-forward transformation. This makes multi-view fusion implicit in text-to-image attention rather than through a separate cross-view transformer on the vision side [2507.17234].

## 3. Section-aware pretraining and structured report representation

Section-aware pretraining is the mechanism that teaches the model both the existence of report sections and the internal granularity of Findings. The report text is rewritten so that each Findings sentence is delimited by a special `<next>` token and the Impression section is introduced by a special `<impression>` token [2507.17234]. In effect, the decoder learns that `<next>` means “continue with another Findings sentence,” whereas `<impression>` means “switch to Impression style and content.”

A typical transformed sequence has the form:

```text
Findings: <next> sentence_1 <next> sentence_2 <next> ... <impression> sentence_impression_1 ...
```

The pretraining objective is standard autoregressive cross-entropy conditioned on the visual sequence [2507.17234]:

\[
\mathcal{L}_{\mathrm{CE}} = -\sum_{t=1}^T \log P_\theta\big(y_t \mid y_{<t}, \widehat{\mathbf{S}}, \mathbf{A}\big)
\]

Here \(y_t\) includes ordinary wordpieces as well as the section-control tokens. The paper does not introduce separate loss terms for Findings and Impression; instead, section structure is imposed through tokenization and sequence organization. This is significant because it gives the model explicit control points later reused at inference time.

The training pipeline includes several implementation decisions that reinforce robustness: images are resized to \(512\times512\), random rotation of \(\pm 5^\circ\) is applied, the number of input images per study is capped at 3, and the order of Findings sentences is shuffled per epoch [2507.17234]. This suggests that section-aware learning is coupled with mild augmentation and structural variability rather than rigid templating.

## 4. Reinforcement learning and Impression-centric reward design

After cross-entropy pretraining, CLARIFID is fine-tuned with **Proximal Policy Optimization (PPO)**. The report generation process is treated as an episode in which the state is the current partial text plus fixed visual features, the action is the next token, and the episode terminates at `<eos>` [2507.17234]. The central design choice is that the reward is computed **only from the Impression section**.

The PPO clipped objective is given as [2507.17234]:

\[
\mathcal{L}_{\text{PPO}}(\theta) = \mathbb{E}_t \left[ \min\Big(r_t(\theta)\,\hat{A}_t,\; \text{clip}(r_t(\theta), 1-\epsilon, 1+\epsilon)\,\hat{A}_t\Big) \right]
\]

with

\[
r_t(\theta) = \frac{\pi_\theta(a_t \mid s_t)}{\pi_{\theta_{\text{old}}}(a_t \mid s_t)}
\]

The reward is the **micro-averaged CheXbert F1** computed over 14 labels on the generated and reference Impression sections. If \(y \in \{0,1\}^{14}\) is the ground-truth label vector and \(\hat y\) is the generated Impression label vector, then [2507.17234]:

\[
P = \frac{TP}{TP+FP},\quad
R = \frac{TP}{TP+FN},\quad
F1 = \frac{2PR}{P+R}
\]

and the scalar episode reward is

\[
R = F1(\text{CheXbert labels of generated Impression},\ \text{ground truth})
\]

Training uses a KL penalty coefficient of **0.05**, reward normalization, and **4 rollouts per sample**. The PPO clip range is \(\epsilon = 0.2\). The PPO learning rate starts at \(1\times10^{-6}\), warms up to \(1\times10^{-5}\) in the first 300 iterations, and remains constant afterward until iteration 1275; batch size is 2 with gradient accumulation 128, rollout temperature is \(T=1.0\), and implementation uses the HuggingFace TRL library [2507.17234].

The reward does not directly score Findings. The paper’s interpretation is that optimizing Impression correctness should implicitly improve the mapping from Findings to diagnosis. The reported training curves support that claim: over **1275 PPO iterations**, Impression CE F1 improves from approximately **0.416 to 0.479**, while Findings CE F1 improves from approximately **0.493 to 0.507** [2507.17234]. This suggests that summary-level clinical supervision can regularize upstream descriptive content when section structure is already explicit.

## 5. Reasoning-aware decoding and inference-time control

At inference time, CLARIFID imposes a two-phase decoding regime. The paper explicitly treats **Findings as the “chain of thought”** and **Impression as the final answer**, and it uses this interpretation to constrain decoding [2507.17234]. The key mechanism is **next-token forcing**: before the model is allowed to emit `<impression>` or `<eos>`, it must generate at least \(k\) `<next>` tokens, corresponding to at least \(k\) Findings sentences.

In the reported configuration, the threshold is **\(k=10\)**, and this value yields the best CE F1 scores for both sections [2507.17234]. The decoding procedure is:

1. Start with “Findings:”.
2. While the number of `<next>` tokens is less than \(k\), sample at temperature \(T_{\text{find}}\); if the model attempts `<impression>` or `<eos>`, override it with `<next>`.
3. After at least \(k\) `<next>` tokens, append `<impression>`.
4. Continue sampling the Impression until `<eos>`.

The inference configuration uses **best-of-\(N\)** decoding with **\(N=8\)** candidate reports, **\(T_{\text{find}}=1.0\)**, **\(T_{\text{imp}}=0.8\)**, and nucleus sampling with **\(p=0.9\)** [2507.17234]. Each candidate report is scored by the value network using the last scalar output \(v_j\), and the final report is selected by

\[
j^\* = \arg\max_j v_j,\quad \hat{s} = \hat{s}_{j^\*}
\]

This is a learned report-level re-ranking mechanism rather than external metric-based selection. No explicit CheXbert scoring is used during inference; the value model acts as a proxy for expected reward [2507.17234].

The decoding ablations are structurally important. Adding **forced generation** after supervised learning and RL improves Findings CE F1 from **0.507 to 0.525** and Impression CE F1 from **0.479 to 0.482**; adding **best-of-\(N\)** on top of that yields the full CLARIFID configuration with Findings CE F1 **0.530** and Impression CE F1 **0.496** [2507.17234]. A plausible implication is that training-time alignment and test-time structural control are complementary rather than interchangeable.

## 6. Empirical performance, ablations, and limitations

The central empirical claim is that CLARIFID improves **clinical efficacy** more than generic text-overlap quality. On **MIMIC-CXR (Findings-only setting)**, CLARIFID achieves **Findings CE F1 = 0.547**, exceeding all listed baselines, including multi-view systems such as **MLRG (0.505)** and **EKAGen (0.499)**, while posting a lower **BLEU-4 = 0.057** than MLRG’s **0.158** [2507.17234]. This establishes the paper’s main trade-off: higher clinically aware scores can coexist with lower n-gram similarity.

On **MIMIC-CXR Impression subset**, where both Findings and Impression are evaluated, CLARIFID reports [2507.17234]:

- **Findings CE**: \(P=0.506\), \(R=0.557\), \(F1=0.530\)
- **Impression CE**: \(P=0.458\), \(R=0.540\), \(F1=0.496\)
- **NLG**: BLEU-1 \(=0.325\), BLEU-4 \(=0.073\), METEOR \(=0.179\), ROUGE-L \(=0.207\)

The paper states that this is the best performance among compared systems for both Findings and Impression clinical efficacy. It also reports that **R2GenGPT (Deep)†** has higher BLEU-4 (**0.135**) but lower Findings CE F1 (**0.360**) and Impression CE F1 (**0.379**) [2507.17234]. The comparison is therefore explicitly between text overlap and section-level clinical faithfulness.

On **IU X-ray**, with training performed only on MIMIC-CXR, CLARIFID achieves **Findings CE F1 = 0.279** and **Impression CE F1 = 0.638**, compared with **0.211** Impression CE F1 for R2GenGPT [2507.17234]. The paper interprets this as strong cross-dataset generalization for the Impression section despite modest BLEU-4 (**0.050**).

The ablation study isolates four configurations on M-CXR Imp. [2507.17234]:

| Configuration | Findings CE F1 | Impression CE F1 |
|---|---:|---:|
| SL only | 0.493 | 0.416 |
| + RL (PPO) | 0.507 | 0.479 |
| + Forced Generation | 0.525 | 0.482 |
| + Best-of-N (full CLARIFID) | 0.530 | 0.496 |

These results show that each component contributes incrementally, and that the largest single gain in Impression CE F1 comes from the addition of reinforcement learning. The forced-generation threshold study further reports that performance improves as required `<next>` tokens increase from 2 to approximately 10 and then saturates, while a temperature grid search identifies \(T_{\text{find}}=1.0\) and \(T_{\text{imp}}=0.8\) as the best pair for Impression F1 [2507.17234].

The paper also states several limitations. Evaluation relies on automatic metrics only; there is **no prospective, blinded clinician evaluation**. CheXbert is acknowledged as an imperfect proxy limited to **14 common diseases**, so optimization may bias the model toward its label space and away from rarer findings. BLEU/ROUGE remain lower than in some text-centric baselines, which may matter for clinician satisfaction. The model is trained only on chest X-rays, and extension to CT, MRI, or other modalities is not studied. Regulatory and ethical questions, including accountability for erroneous reports, are not addressed [2507.17234].

Taken together, CLARIFID defines report generation as a workflow-aligned, section-structured, clinically supervised generation problem rather than a generic sequence modeling task. Its core contribution is not a new language model family but the alignment of architecture, reward, and decoding with the radiologist’s observations-to-impression pipeline, yielding state-of-the-art clinical efficacy on the reported benchmarks [2507.17234].

Source: https://www.emergentmind.com/topics/clarifid