Enhancing Fine-Grained Spatial Grounding in 3D CT Report Generation via Discriminative Guidance

Abstract: Vision--LLMs (VLMs) for radiology report generation (RRG) can produce long-form chest CT reports from volumetric scans and show strong potential to improve radiology workflow efficiency and consistency. However, existing methods face two key limitations: (i) training supervision is often coarse, aligning a whole CT volume with a full free-text report without explicit alignment for fine-grained attributes or pathology locations; and (ii) evaluation is typically holistic (lexical overlap, entity matching, or LLM-as-a-judge scores) and not diagnostic for spatial grounding. We propose \emph{Discriminative Cue-Prompting with Prompt Dropout (DCP-PD)}, a plug-and-play framework that distills fine-grained cues from free-text reports and uses them to guide report generation while mitigating shortcut reliance via prompt dropout. DCP-PD achieves state-of-the-art performance on CT-RATE, improving macro F1 from $=0.501$ to $0.603$ (20% relative), and substantially boosts out-of-distribution performance on Rad-ChestCT from F1 $=0.266$ to $0.503$ (89% relative). Finally, we introduce a hierarchical, location-aware question-set protocol (presence $\rightarrow$ laterality $\rightarrow$ lobe) to directly assess pathology-location grounding, showing that fine-grained spatial localization remains challenging even for models that score highly on current benchmarks.

• The paper introduces DCP-PD, a novel modular framework leveraging discriminative cues and prompt dropout to improve fine-grained spatial grounding in CT report generation.
• It integrates a dual-stage training approach that aligns multi-scale CT features with language, significantly boosting pathology detection and localization metrics.
• Experimental results show state-of-the-art performance on CT-RATE and improved generalization on OOD data, underscoring its clinical potential.

Enhancing Fine-Grained Spatial Grounding in 3D CT Report Generation via Discriminative Guidance

Motivation and Problem Statement

Current VLMs for 3D chest CT report generation achieve strong results in structured narrative creation but consistently underperform on fine-grained abnormality detection and precise pathology-location grounding when compared to standalone discriminative classification models. Critically, conventional training paradigms enforce only volumetric-level image-text alignment, neglecting explicit supervision for localization or attribute-level concepts. Furthermore, standard evaluation metrics (BLEU, METEOR, entity F1, LLM-judge) are insufficiently granular for assessing whether the generated narrative is accurately spatially grounded. This creates an urgent need for architectures and protocols that offer diagnostic fine-grained supervision and allow modular extension to new spatial attributes without entangling the VLM with auxiliary discriminators.

DCP-PD Framework and Methodology

The proposed DCP-PD (Discriminative Cue-Prompting with Prompt Dropout) framework is a paradigm shift for modular, spatially explicit 3D chest CT report generation. The method first restructures raw free-text reports into sectioned, semantically organized formats using LLM-driven parsing, extracting clinically relevant attributes by templated, question-driven binary queries (e.g., presence, laterality, lobar localization). This question-driven extraction enables the construction of hierarchical supervision at three diagnostic levels: global finding presence, laterality (left/right), and lobe assignment.

Figure 1: Pipeline overview—report structuring, label extraction with hierarchical clinical queries, multi-scale CT feature extraction, cue prompt formation, and decoupled VLM training with discriminative guidance and prompt dropout.

Using frozen multi-scale visual representations (Atlas backbone with MSA), lightweight multi-label discriminators are trained per-question set. These discriminators are decoupled from the VLM, and their predictions are templated into natural-language cue prompts.

The report generator consists of a dual-stage training protocol:

1. Alignment Stage: The visual projector is tuned (LLM frozen) for aligning vision features to language space.
2. Discriminative-Guided SFT Stage: The vision projector and LoRA-tuned LLM are jointly trained; at each iteration, the generator receives ground-truth cue prompts with independently applied prompt dropout (randomly omitting cues) to mitigate shortcut copying and enforce strong grounding in visual evidence.

At inference, cues are produced by any discriminative model, preserving modularity and upgradability of the auxiliary model without retraining the VLM.

Figure 2: Atlas-based multi-scale encoding; cue generation by discriminators; VLM generates reports conditioned on concatenated image tokens and discriminative cue tokens.

Experimental Evaluation

Benchmarking and Clinical Metrics

DCP-PD is evaluated primarily on CT-RATE (in-distribution) and Rad-ChestCT (OOD, external validation). Report quality is measured not only via lexical metrics (BLEU, METEOR) but also using granular clinical correctness scores (macro-F1 for 18 findings, laterality, and lobe), and distribution-aware CRG.

• Discriminator superiority: Linear-probe discriminators outperform base VLMs on finding-level classification (ΔF1 up to 27%, Figure 3).
• Model-agnostic improvements: Conditioning the VLM on classifier predictions (DCP-1, DCP-2) boosts report-level F1 by ~20–37% over non-cue-guided baselines.

  Figure 3: Per-pathology F1; discriminators markedly outperform the base VLM in almost all findings.

• Prompt dropout ablation: Absence of prompt dropout results in the VLM collapsing (shortcut copying) when cues are omitted at test time (F1 drops to 0.094). Use of DCP-PD enforces robust visual dependence (F1 recovers to 0.366 without cues).
• Attention analysis: Prompt dropout reduces attention mass on cue tokens ($S_\text{text}$ decreases from 0.441 to 0.316), confirming increased visual grounding.

Comparison to SOTA and Generalization

The DCP-PD-guided VLM attains state-of-the-art performance on CT-RATE across clinical and lexical metrics and demonstrates robust generalization on Rad-ChestCT (OOD F1 increases from 0.266 to 0.503).

Figure 4: Per-pathology F1 on CT-RATE; DCP-PD surpasses all baselines on every finding.

Fine-Grained Spatial Grounding

DCP-PD’s modularity allows seamless extension to laterality and lobe cues. At each refinement level, F1 is consistently increased over the base VLM and approaches (sometimes matches or surpasses) standalone classifier performance for localizable findings.

Figure 5: Laterality-level (left/right) F1 across common findings; DCP-PD closes the gap to discriminative classification.

Figure 6: Lobe-level F1; DCP-PD improves over the base VLM and parallels classifier accuracy for most localization tasks.

Removal of image tokens (cue-only LLM) maintains strong coarse pathology reporting but fails at location-level grounding, underscoring the essential value of visual input for DCP-PD's spatial fidelity.

Figure 7: Visual tokens are critical for fine-grained lobar assignment, as shown by ΔF1 (VLM–LLM); purely cue-conditioned LLMs underperform in spatial tasks.

Qualitative Analysis

Qualitative comparison on clinical samples demonstrates that DCP-PD not only detects findings missed by the baseline VLM but also yields more spatially accurate, structured narratives specifically when guided by discriminative location cues.

Figure 8: Case study—DCP-PD recovers missed abnormalities and more precise anatomical localization compared to an unguided VLM.

Figure 9: DCP-PD produces full-structured reports with faithful pathology–location associations, absent in baseline models.

Implications and Future Directions

DCP-PD establishes a protocol for modular, interpretable, and extensible radiology report generation that is decoupled from auxiliary discriminators. The ability to swap, upgrade, or specialize the discriminator (for new pathologies or attributes) without retraining the VLM introduces new possibilities for scalable clinical deployment. The prompt dropout mechanism is essential for eliminating shortcut failures, ensuring the LLM does not merely replicate prompt cues but remains tightly grounded in the visual evidence.

Moreover, the hierarchical, question-driven evaluation reveals that fine-grained spatial grounding remains a partially unsolved challenge—even SOTA VLMs may fail in attribute-level or regional assignments. Adoption of DCP-PD-like methods and evaluations will likely become a requirement for regulatory-grade medical AI systems, addressing clinical safety concerns around hallucinated or mislocalized findings.

Upcoming research directions include expanding support for additional attribute cues (severity, progression), richer structured narratives, discriminator ensemble conditioning, and prospective multicenter clinical validation—especially for rare and low-prevalence findings.

Conclusion

DCP-PD represents a modular and effective framework for explicitly guided, attribute-hierarchical CT report generation with superior pathology detection and localization performance. Disentangling VLMs from specific discriminators and using cue-prompting with robust anti-shortcut mechanisms results in clinically safer and more extensible AI report generators. This approach is critical for bridging the gap between coarse holistic evaluation and true spatially faithful clinical narratives in radiological AI.