---
title: 'Report2CT: CT Report-Conditioned Generation'
url: https://www.emergentmind.com/topics/report2ct
type: topic
---

# Report2CT: CT Report-Conditioned Generation

Searching arXiv for recent papers directly related to “Report2CT” and closely adjacent CT report generation work.
I’ll look for arXiv entries on “Report2CT” and relevant CT report generation benchmarks and methods.
Report2CT denotes a line of CT-language modeling in which radiology reports are treated as primary conditioning signals, intermediate clinical scaffolds, or evaluation anchors for computed tomography systems. In its most specific usage, Report2CT is a radiology report conditional latent diffusion framework for synthesizing full 3D chest CT volumes directly from free-text radiology reports, incorporating both findings and impression sections using multiple text encoder [2509.14780]. In adjacent work, the label “Report2CT-style” is used for clinically grounded pipelines that connect CT slices or volumes, structured abnormalities, and report text rather than treating the problem as unconstrained captioning [2506.23584]. Taken together, these usages place reports at the center of CT synthesis, CT report generation, cross-modal transfer, weak supervision, and diagnostic evaluation.

## 1. Conceptual scope

The Report2CT formulation emerged from a common observation: simplified prompts or global image summaries are inadequate for CT, because CT descriptions are dense with lesion-specific attributes such as location, size, margin, density, shape, pattern, and type [2604.24001]. In the synthesis setting, the argument is that a full radiology report contains findings and impression, including location, morphology, extent, severity, normal anatomy, and concise expert interpretation, whereas a prompt such as “pneumonia” discards laterality, lobe-level location, coexistence of findings, and subtle descriptors like “spiculated,” “occlusive,” “minimal,” or “consistent with” [2509.14780]. In the report-generation setting, a similar argument appears in renal CT: reporting is not just “image captioning,” because it requires identifying subtle lesion properties, some of which are often missing or inconsistently documented in real clinical reports [2506.23584].

This shared motivation differentiates Report2CT from purely lexical or purely visual approaches. A plausible implication is that the report is being used not merely as text output, but as a clinically structured representation of CT content. That interpretation is reinforced by methods that first infer structured lesion descriptors and then generate report text, by methods that use reports as a semantic bridge between CT and chest radiography, and by evaluation frameworks that score whether a generated report preserves fine-grained diagnostic facts rather than whether it resembles a reference at the n-gram level [2503.02162].

## 2. Report-conditioned 3D CT synthesis

The model explicitly named Report2CT is an end-to-end 3D latent diffusion framework for text-conditional synthesis of full 3D chest CT volumes from radiology reports [2509.14780]. The pipeline comprises CT volume preprocessing, image compression, text encoding, conditional latent diffusion, and decoding. Input CTs are resampled to **480 × 480 × 256**, intensities are clipped to **[-1000, 1000] HU**, then min-max normalized to **[0, 1]**, and voxel spacing changes caused by resampling are explicitly encoded as conditioning information. A pretrained **MAISI AutoencoderKL** compresses each 3D CT from `480 × 480 × 256` into a latent tensor of `4 × 120 × 120 × 64`, after which a **3D U-Net diffusion model** learns to denoise latent CT representations with conditioning injected through **cross-attention** [2509.14780].

A distinguishing feature is multi-encoder text conditioning. The report is split into **findings** and **impression**, each section is encoded separately by **BiomedVLP-CXR-BERT**, **MedEmbed**, and **ClinicalBERT**, masked mean pooling is applied, and pooled vectors from the three encoders are concatenated. The resulting representation is **768 + 768 + 1024 = 2560 dimensions** for each section, and the findings and impression embeddings are concatenated channel-wise to form the final text-conditioning tensor. The model is trained on **20,000 CT volumes** from the **CT-RATE** dataset; the MAISI compression network was pretrained on **39,206 3D CT volumes** [2509.14780].

Training uses **2 NVIDIA H100 NVL GPUs**, distributed data parallel training, mixed precision, **batch size: 2**, **learning rate: 1e-4**, **Adam**, a polynomial decay scheduler, and **Rectified Flow (RFlow)** with **1,000 steps**. Classifier-free conditioning dropout is **0.15**, enabling classifier-free guidance at inference. Evaluation uses **Fréchet Inception Distance (FID)** with a **2.5D feature extraction strategy** based on a fixed **RadImageNet-pretrained ResNet-50**, and **CLIPScore-T2I** plus **CLIPScore-I2I** for semantic alignment. The paper reports that multi-encoder conditioning improved CLIP scores, that classifier-free guidance further enhanced alignment with only a minor trade-off in FID, and that the method ranked **1st** in **Task 4 of the VLM3D Challenge at MICCAI 2025** on **Text-Conditional CT Generation** [2509.14780].

## 3. Structured Report2CT-style report generation

A clinically grounded “two-stage Report2CT-style framework” appears in renal CT report generation, where the task is to generate renal radiology report sentences from **2D CT slices** [2506.23584]. The method separates the problem into structured abnormality understanding and text generation. The curated dataset was built from UF Health clinical data by filtering renal-related CPT-9 codes **74160, 74170, 74175, 74177, 74178**, matching reports to CT scan records, retaining only reports that explicitly referenced a CT slice, typically in the **coronal view**, and excluding cases with no abnormalities, missing CT series, or non-coronal slice references. After manual review, the dataset contains **130 annotations**, from **108 reports**, across **97 patients**, organized as sentence–slice–feature triplets:
\[
(\text{report sentence}, \text{renal feature labels}, \text{CT slice})
\]
[2506.23584].

Stage 1 uses a **ResNet-34** encoder, with grid search over **ResNet-18, ResNet-50, and ResNet-101**, to predict eight renal feature labels: **Position**, **Enhancement**, **Cyst**, **Mass**, **Attenuation**, **Size**, **Tumor**, and **Exophytic**. Training uses **5-fold cross-validation**, **stratified sampling**, input slices resized to **512 × 512**, CT intensity clipping with **window width 400, window level 50**, min–max normalization to **[-1, 1]**, adjacent slices at offsets **−1 and +1**, **batch size = 16**, **learning rate = 1e-4**, and **Adam optimizer**. Representative results include **Position: AUC 0.6664, F1 0.7050**, **Enhancement: AUC 0.8500, F1 0.8859**, **Cyst: AUC 0.7615, F1 0.7607**, **Mass: AUC 0.6094, F1 0.6211**, **Tumor: AUC 0.7063, F1 0.7400**, **Attenuation: Accuracy 0.6827, F1 0.6167**, and **Size: MSE 0.9959 cm** [2506.23584].

Stage 2 fine-tunes **Qwen2.5-VL-7B-Instruct** with **LoRA** using **LLAMA-Factory**. The model receives a structured feature template plus the CT slice image and learns the mapping
\[
\hat{y} = f_{\theta}(I, \mathbf{z}),
\]
where \(I\) is the CT slice and \(\mathbf{z}\) is the structured feature vector or template. Training uses **LoRA rank = 8**, **learning rate = 1e-4**, **batch size = 4**, **gradient accumulation = 8**, a **cosine scheduler**, **200 epochs**, **bf16 mixed precision**, evaluation every 50 steps, and the same **window width = 400**, **window level = 50**, and **512 × 512** resizing. Across 5-fold cross-validation, report generation achieved **BLEU-4 = 29.58**, **ROUGE-1 = 28.31**, **ROUGE-2 = 9.59**, and **ROUGE-L = 21.99** [2506.23584].

This two-stage design is echoed, with different architectural choices, by 3D CT report generators that make structure or region evidence explicit: **CT2Rep** introduced the first reported framework for automated radiology report generation from **3D medical imaging**, specifically non-contrast chest CT volumes [2403.06801]; **Reg2RG** proposed the first region-guided referring and grounding framework for CT report generation [2411.15539]; **CT-GRAPH** organized organ and organ-system features into a hierarchical graph [2508.05375]; **MedRegion-CT** used **Region Representative ($R^2$) Token Pooling**, pseudo-masks, and patient-specific attributions [2506.23102]; and **Structure Observation Driven Image-Text Contrastive Learning for Computed Tomography Report Generation** treated CTRG as a structure-centric vision-language learning problem with structure-specific visual queries [2603.04878].

## 4. Region, structure, and retrieval as Report2CT design patterns

Subsequent CT report-generation systems broadened the Report2CT-style design space by making anatomical structure, region grounding, or retrieved text explicit. **Reg2RG** uses anatomical masks from a universal segmentation module, local feature decoupling into texture and geometry, region-report alignment with shuffled local features, and **LLaMA2-7B** with **LoRA**; on **RadGenome-ChestCT**, it achieved **Precision: 0.423**, **Recall: 0.181**, and **F1: 0.253** for clinical efficacy [2411.15539]. **CT-GRAPH** builds a hierarchy of fine-level nodes, coarse anatomical-system nodes, and a global node from frozen pretrained 3D encoder features and anatomical masks; on **CT-RATE**, it reports **CE F1 = 0.296**, compared with **CT2Rep w/ LLaMA: CE F1 = 0.214** and **Reg2RG: CE F1 = 0.217**, and the abstract highlights an absolute **7.9\%** improvement in F1 [2508.05375].

**MedRegion-CT** is a region-focused MLLM using a frozen **RAD-DINO ViT-B** vision encoder, **LLaMA3-8B**, pseudo-masks from **SAT**, six predefined chest regions, and deterministic extraction of organ volume, lesion count, lesion diameter, and lesion location from masks. On **RadGenome-Chest CT**, it reports **BLEU-4: 0.290**, **ROUGE-L: 0.375**, **METEOR: 0.494**, **CA-F1: 0.450**, **Green score: 0.278**, and **GPT-4 score: 48.837** [2506.23102]. **AdaRAG-CT** reframes report generation as a response to an embedding bottleneck in contrastive 3D CT encoders, noting that CT-CLIP avg pool has \( \text{dim}_{90} = 2 \) out of 512 and participation ratio \( \text{PR} = 1.4 \); it adds an adaptive retrieval channel with a learned **[RAG]** token and improves **Clinical F1** on **CT-RATE** from **0.420 (CT-Agent)** to **0.480** [2603.15822]. **Ker-VLJEPA-3B** instead uses a frozen, self-supervised **LeJEPA ViT-Large**, **zone-constrained cross-attention** that compresses slice embeddings into **32** spatially grounded visual tokens, and a four-phase curriculum; on **CT-RATE**, it reports **macro F1 = 0.429**, reaching **0.448** with threshold optimization [2603.23308].

These systems differ substantially in mechanism, but they share a common Report2CT-style premise: global volume embeddings alone are usually insufficient for fine-grained CT reporting. This suggests that Report2CT has evolved from direct vision-to-language mapping toward architectures that expose anatomical regions, structure-aware tokens, or supplementary textual evidence to the generator.

## 5. Evaluation and diagnostic fidelity

Evaluation has become a central problem for Report2CT research because conventional metrics are often too coarse for CT reports. **CT-FineBench** was proposed as a benchmark for fine-grained factual consistency of CT reports, built from **CT-RATE** and **Merlin** through a Question-Answering-based process that first identifies clinically critical attributes and then converts them into QA pairs [2604.24001]. The final benchmark contains **more than 44k QA pairs** derived from **6,646 reports**, and the companion training corpus **CT-FineData** contains **over 439k QA pairs**. Its scoring rule is **\{0, 0.5, 1\}**, with synonym-aware exact match for categorical attributes and relative-error thresholds for numeric attributes. In human-correlation experiments, two medical experts rated sampled generated reports on a 10-point factual accuracy scale, and on **CT-RATE** CT-FineBench reached the highest Pearson correlation at **0.622** [2604.24001].

A complementary benchmark addresses abdominal Report2CT-style generation from non-contrast CT. **A Multi-Center Benchmark for Abdominal Disease Diagnosis and Report Generation from Non-Contrast CT** curated a retrospective multi-center cohort of **1,254 patients** from **two tertiary institutions**, with **Center 1 (internal cohort): 1,085 patients/volumes** and **Center 2 (external validation cohort): 169 patients/volumes** [2606.16991]. The dataset supports multi-label disease classification across **53 abdominal pathologies** spanning **15 organs**, and free-text report generation from NCCT to contrast-enhanced report targets. Under this benchmark, the abstract reports an average multi-organ AUC of **69.1%** on the internal cohort and **63.1%** on the external cohort. For report generation, the paper explicitly prioritizes **GREEN** and **RadGraph-XL** over **BLEU-4**, **METEOR**, **BERTScore**, and **ROUGE-L**, arguing that standard text-overlap metrics are not enough in radiology [2606.16991].

For Report2CT systems, these developments mean that evaluation is increasingly moving from surface similarity to attribute-level factual consistency, omission detection, and clinically oriented entity-relation scoring. A plausible implication is that future Report2CT work will be judged less by fluency alone and more by whether it preserves lesion location, lesion size, margin, density, and type.

## 6. Report supervision, transfer, and interactive extensions

Report2CT has also expanded beyond direct report generation or report-conditioned synthesis. **X2CT-CLIP** uses a tri-modal latent space comprising CT volume \(C\), CT report \(R\), and chest radiography \(X\), where the report functions as the semantic intermediary that transfers CT knowledge into a CXR encoder [2503.02162]. CT-CLIP’s pretrained CT and report encoders are frozen, only the CXR encoder is trained, and the actual setting uses **\(\beta = 1\)**, **\(\gamma = 1\)**, and **\(\alpha = 0\)** in the tri-modal objective. On **CT-RATE**, **RadChest-CT**, and **MIMIC-CT**, the paper reports zero-shot AUC examples of **0.716**, **0.645**, and **0.567**, respectively [2503.02162]. Here the report is neither label text nor generated output; it is the alignment anchor.

**Learning Segmentation from Radiology Reports** moves further toward voxel-wise supervision from text. The method, **R-Super**, uses **Llama 3.1 70B AWQ** to extract tumor presence, location, diameters, and count from reports, achieving about **96% accuracy** for tumor presence/location extraction, and converts those attributes into **Volume Loss** and **Ball Loss** for CT tumor segmentation [2507.05582]. The private dataset **UCSF-Train** contains **6,718 CT–Report pairs**, combined with public CT-mask datasets from **AbdomenAtlas 2.0**. The abstract reports that **F1 Score increased by up to 16%** with respect to training with masks only [2507.05582].

A different extension is interactive lesion morphology reporting. **Interactive Segmentation and Report Generation for CT Images** jointly generates segmentation masks and a structured report with **five lesion morphology attributes**—**Shape**, **Invasion**, **Density**, **Heterogeneity**, and **Surface**—from 3D CT medical images [2503.03294]. On **1535 CT scans and masks** across **15 lesion types**, the full model reports a **test set average accuracy** of **0.759** for structured report generation and **0.794 DSC / 4.303 HD95** for segmentation, with **gallstone** and **liver cyst** treated as zero-shot test cases [2503.03294]. Although this is not free-form report generation, it shows that Report2CT-style ideas can be instantiated as interactive, visually grounded structured reporting.

## 7. Limitations and open directions

The current Report2CT literature repeatedly identifies data scarcity, imbalance, and evaluation mismatch as limiting factors. In renal CT, the dataset is modest and highly imbalanced, many reports do not mention certain features, and the system uses one slice per sample, losing 3D lesion extent, cross-slice spatial continuity, and volumetric context; the paper therefore identifies extension to **3D CT volumes** as the main future direction [2506.23584]. In report-conditioned 3D CT synthesis, the method is computationally demanding, depends on detailed reports with findings and impression, and does not compare directly with Text2CT because of lack of public code or paired synthetic outputs [2509.14780].

Generalization remains difficult. The multi-center abdominal NCCT benchmark reports substantial external drops and explicitly notes that enhancement-kinetic findings such as arterial hyperenhancement, portal venous washout, delayed retention, and capsule appearance cannot truly be recovered from NCCT, so an information ceiling remains [2606.16991]. In retrieval-augmented report generation, AdaRAG-CT characterizes itself as a workaround rather than a root-cause fix, because the retrieval database is built entirely from the training corpus and the true limitation lies in impoverished visual representations [2603.15822]. Evaluation is also unresolved: CT-FineBench is especially good at detecting omissions because questions are constructed from facts present in the reference report, but it is less suited to penalizing hallucinated findings not anchored in the reference [2604.24001].

Across these studies, the dominant research directions are consistent: extension from 2D slices to 3D volumes, larger and better balanced datasets, domain-specific pretraining, more explicit grounding and localization, clinically grounded evaluation beyond BLEU and ROUGE, better external generalization across institutions, and tighter integration between report generation and CT generation. This suggests that Report2CT is evolving from a narrowly defined model name into a broader research program centered on clinically faithful report-conditioned and report-grounded CT intelligence.

Source: https://www.emergentmind.com/topics/report2ct