Precision Recall Controllable Radiology Report Generation via Hybrid Natural Language and Clinical Reward Learning

Abstract: Automated radiology report generation (RRG) has gained increasing attention because it can reduce the heavy workload of clinical report writing. However, most existing methods mainly optimize for natural language generation (NLG) metrics that focus on language fluency, while providing little control over clinically important factors such as precision and recall. As consequence, generated reports may be fluent but not well aligned with different clinical needs. To address this challenge, we propose a reinforcement learning framework for precision recall controllable RRG, where a control parameter explicitly adjusts the trade-off between clinical precision and recall during inference. This design allows the model to flexibly generate reports according to different clinical requirements. To ensure clinical correctness, we introduce a clinical reward into the training objective, which helps improve clinical efficacy (CE) beyond standard language-based optimization. In addition, we apply a group-relative training strategy that normalizes rewards within each training group, reducing reward variance and improving training stability. Extensive experiments on the MIMIC-CXR dataset show that our method consistently outperforms state-of-the-art approaches in both NLG and CE evaluation metrics, while providing reliable control over the CE precision recall trade-off.

• The paper introduces a hybrid RL framework that enables continuous control of the precision-recall trade-off in radiology report generation through a λ parameter.
• It integrates reinforcement learning with cross-entropy loss and employs group-relative training to optimize both linguistic quality and clinical efficacy.
• Experiments on the MIMIC-CXR dataset demonstrate superior performance over baselines, achieving a CE F1 score of 0.571 and enhanced report fidelity.

Precision Recall Controllable Radiology Report Generation via Hybrid Natural Language and Clinical Reward Learning

Introduction

Automated radiology report generation (RRG) is central to reducing clinical workload and scaling medical imaging analysis. Prevailing RRG methods primarily optimize natural language generation (NLG) metrics, favoring grammatical fluency over clinical relevance. This approach yields reports with limited clinical efficacy (CE) and provides minimal control over trade-offs between precision and recall—variables that are highly context-dependent in actual clinical workflows. The paper introduces a reinforcement learning (RL) framework that allows explicit, continuous modulation of the precision-recall balance using a control parameter, $\lambda$, fundamentally restructuring the optimization target for both linguistic and clinical objectives.

Figure 1: Overall architecture, RL process, and transformer decoder highlighting representation and reward-level precision-recall control via $\lambda$ and group-relative training.

Methodology

Precision-Recall Control via Reinforcement Learning

The model parameterizes precision-recall control through a continuous variable $\lambda \in [0,1]$, integrated at both the representation and reward levels. At the representation level, the Precision Recall Conditioned Adaptive Layer Normalization (PRC-AdaLN) module leverages conditional normalization (akin to FiLM), enabling $\lambda$-conditioned modulation of transformer decoder statistics. At the reward level, $\lambda$ serves as a direct weight between precision and recall CE rewards:

$$R_{\mathrm{CE}}(\lambda) = \lambda \, R_{\mathrm{prec}} + (1 - \lambda) \, R_{\mathrm{rec}}$$

Joint sampling of $\lambda$ during training produces a family of policies, permitting flexible inference-time adjustment without retraining.

Hybrid Reward Formulation

Report generation is optimized via a hybrid loss comprising RL and cross-entropy components, weighted by $\alpha = 0.99$:

$$\mathcal{L}_{\text{total}} = \alpha \mathcal{L}_{\text{rl}} + (1 - \alpha)\mathcal{L}_{\text{ce}}$$

RL loss is computed using policy sampling and the hybrid reward:

$$R(Y) = \rho R_{\text{NLG}}(Y) + (1 - \rho) R_{\text{CE}}(Y)$$

where $\lambda$0 aggregates BLEU-4, METEOR, and ROUGE-L, and $\lambda$1 is precision-recall balanced by $\lambda$2. This hybrid mechanism ensures robust linguistic quality alongside clinically meaningful content.

Group-Relative Training

Training stability is enhanced through group-relative self-critical sequence training: for each input, $\lambda$3 sampled reports are normalized, yielding standardized advantages which replace absolute rewards in policy gradient optimization. This scheme reduces reward variance and stabilizes convergence, particularly given the multi-reward scenario necessitated by the hybrid objective.

Experiments

Dataset and Metrics

The MIMIC-CXR dataset offers a large-scale, patient-level split benchmark, focusing on the findings section of reports. Evaluation employs both NLG (BLEU, METEOR, ROUGE-L) and CE metrics (micro-averaged precision, recall, F1 over CheXbert labels).

Precision-Recall Control Analysis

Comprehensive ablation demonstrates effective, continuous modulation of clinical precision and recall via $\lambda$4. As $\lambda$5 increases, precision rises and recall falls, forming a trade-off curve with optimal F1 performance at $\lambda$6. This enables deployment versatility matching clinical scenario requirements.

Figure 2: Clinical precision, recall, and F1-score response to the precision-recall control parameter $\lambda$7.

Performance Benchmarking

Compared against leading methods (R2Gen, METrans, R2GenGPT, BoostRRG, Diff-RRG, MLRG, MedGemma), the proposed framework achieves superior scores on both NLG and CE metrics. Notably, CE F1 reaches 0.571, outperforming all baselines, with substantial improvements in modeling long-range text dependencies.

Ablation Study

The individual and joint effects of clinical reward and group-relative training are evaluated. Clinical reward alone boosts CE metrics, while group-relative training impacts both NLG and CE. The full model leveraging both achieves optimal performance, indicating effective synergy.

Qualitative Analysis

Representative qualitative samples highlight enhanced clinical fidelity. The proposed method captures findings such as hyperinflated lungs and enlarged cardiac silhouette, aligning closely with ground truth and surpassing the baseline in clinical correctness.

Figure 3: Qualitative comparison of generated and ground truth reports; improvements (blue) demonstrate enhanced clinical accuracy and alignment.

Implications and Future Directions

The explicit precision-recall control enables broader deployment of RRG systems in varied clinical workflows, from high-recall screening to high-precision confirmatory contexts. The hybrid reward architecture strengthens clinical accuracy without sacrificing linguistic quality, while group-relative training addresses instability in RL optimization. The framework's modularity allows adaptation to larger-scale LLMs (e.g., MedGemma) and cleaner datasets. Future work includes integration of Group reward-decoupled normalization (GDPO), extension to advanced anatomy-grounded report generation (e.g., CURE), comprehensive metric evaluation, and reader studies. These directions will further bridge the gap between automated and human-generated reports in clinical practice.

Conclusion

The hybrid RL framework for precision-recall controllable radiology report generation constitutes a significant advancement in harmonizing semantic fluency with clinical efficacy. Continuous $\lambda$8-driven modulation, clinical reward integration, and group-relative training collectively produce state-of-the-art report quality and clinical alignment. Future expansions into LLMs and more rigorous clinical evaluation metrics are expected to further solidify the practical impact of these innovations.