Report2CT: CT Report-Conditioned Generation

• Report2CT is a radiology report–conditioned latent diffusion framework that synthesizes full 3D chest CT volumes using multi-encoder text conditioning.
• The approach splits the report into findings and impression sections, leveraging multiple text encoders to capture detailed lesion attributes for enhanced diagnostic accuracy.
• It extends into structured report generation, employing two-stage pipelines that extract lesion features from CT slices to drive clinically grounded radiology report outputs.

Searching arXiv for papers directly related to “Report2CT” and closely adjacent CT report generation work. I’ll look for arXiv entries on “Report2CT” and relevant CT report generation benchmarks and methods. Report2CT denotes a line of CT-language modeling in which radiology reports are treated as primary conditioning signals, intermediate clinical scaffolds, or evaluation anchors for computed tomography systems. In its most specific usage, Report2CT is a radiology report conditional latent diffusion framework for synthesizing full 3D chest CT volumes directly from free-text radiology reports, incorporating both findings and impression sections using multiple text encoder (Amirrajab et al., 18 Sep 2025). In adjacent work, the label “Report2CT-style” is used for clinically grounded pipelines that connect CT slices or volumes, structured abnormalities, and report text rather than treating the problem as unconstrained captioning (Liang et al., 30 Jun 2025). Taken together, these usages place reports at the center of CT synthesis, CT report generation, cross-modal transfer, weak supervision, and diagnostic evaluation.

1. Conceptual scope

The Report2CT formulation emerged from a common observation: simplified prompts or global image summaries are inadequate for CT, because CT descriptions are dense with lesion-specific attributes such as location, size, margin, density, shape, pattern, and type (Yuan et al., 27 Apr 2026). In the synthesis setting, the argument is that a full radiology report contains findings and impression, including location, morphology, extent, severity, normal anatomy, and concise expert interpretation, whereas a prompt such as “pneumonia” discards laterality, lobe-level location, coexistence of findings, and subtle descriptors like “spiculated,” “occlusive,” “minimal,” or “consistent with” (Amirrajab et al., 18 Sep 2025). In the report-generation setting, a similar argument appears in renal CT: reporting is not just “image captioning,” because it requires identifying subtle lesion properties, some of which are often missing or inconsistently documented in real clinical reports (Liang et al., 30 Jun 2025).

This shared motivation differentiates Report2CT from purely lexical or purely visual approaches. A plausible implication is that the report is being used not merely as text output, but as a clinically structured representation of CT content. That interpretation is reinforced by methods that first infer structured lesion descriptors and then generate report text, by methods that use reports as a semantic bridge between CT and chest radiography, and by evaluation frameworks that score whether a generated report preserves fine-grained diagnostic facts rather than whether it resembles a reference at the n-gram level (You et al., 4 Mar 2025).

2. Report-conditioned 3D CT synthesis

The model explicitly named Report2CT is an end-to-end 3D latent diffusion framework for text-conditional synthesis of full 3D chest CT volumes from radiology reports (Amirrajab et al., 18 Sep 2025). The pipeline comprises CT volume preprocessing, image compression, text encoding, conditional latent diffusion, and decoding. Input CTs are resampled to 480 × 480 × 256, intensities are clipped to [-1000, 1000] HU, then min-max normalized to [0, 1], and voxel spacing changes caused by resampling are explicitly encoded as conditioning information. A pretrained MAISI AutoencoderKL compresses each 3D CT from 480 × 480 × 256 into a latent tensor of 4 × 120 × 120 × 64, after which a 3D U-Net diffusion model learns to denoise latent CT representations with conditioning injected through cross-attention (Amirrajab et al., 18 Sep 2025).

A distinguishing feature is multi-encoder text conditioning. The report is split into findings and impression, each section is encoded separately by BiomedVLP-CXR-BERT, MedEmbed, and ClinicalBERT, masked mean pooling is applied, and pooled vectors from the three encoders are concatenated. The resulting representation is 768 + 768 + 1024 = 2560 dimensions for each section, and the findings and impression embeddings are concatenated channel-wise to form the final text-conditioning tensor. The model is trained on 20,000 CT volumes from the CT-RATE dataset; the MAISI compression network was pretrained on 39,206 3D CT volumes (Amirrajab et al., 18 Sep 2025).

Training uses 2 NVIDIA H100 NVL GPUs, distributed data parallel training, mixed precision, batch size: 2, learning rate: 1e-4, Adam, a polynomial decay scheduler, and Rectified Flow (RFlow) with 1,000 steps. Classifier-free conditioning dropout is 0.15, enabling classifier-free guidance at inference. Evaluation uses Fréchet Inception Distance (FID) with a 2.5D feature extraction strategy based on a fixed RadImageNet-pretrained ResNet-50, and CLIPScore-T2I plus CLIPScore-I2I for semantic alignment. The paper reports that multi-encoder conditioning improved CLIP scores, that classifier-free guidance further enhanced alignment with only a minor trade-off in FID, and that the method ranked 1st in Task 4 of the VLM3D Challenge at MICCAI 2025 on Text-Conditional CT Generation (Amirrajab et al., 18 Sep 2025).

3. Structured Report2CT-style report generation

A clinically grounded “two-stage Report2CT-style framework” appears in renal CT report generation, where the task is to generate renal radiology report sentences from 2D CT slices (Liang et al., 30 Jun 2025). The method separates the problem into structured abnormality understanding and text generation. The curated dataset was built from UF Health clinical data by filtering renal-related CPT-9 codes 74160, 74170, 74175, 74177, 74178, matching reports to CT scan records, retaining only reports that explicitly referenced a CT slice, typically in the coronal view, and excluding cases with no abnormalities, missing CT series, or non-coronal slice references. After manual review, the dataset contains 130 annotations, from 108 reports, across 97 patients, organized as sentence–slice–feature triplets: $$(\text{report sentence}, \text{renal feature labels}, \text{CT slice})$$ (Liang et al., 30 Jun 2025).

Stage 1 uses a ResNet-34 encoder, with grid search over ResNet-18, ResNet-50, and ResNet-101, to predict eight renal feature labels: Position, Enhancement, Cyst, Mass, Attenuation, Size, Tumor, and Exophytic. Training uses 5-fold cross-validation, stratified sampling, input slices resized to 512 × 512, CT intensity clipping with window width 400, window level 50, min–max normalization to [-1, 1], adjacent slices at offsets −1 and +1, batch size = 16, learning rate = 1e-4, and Adam optimizer. Representative results include Position: AUC 0.6664, F1 0.7050, Enhancement: AUC 0.8500, F1 0.8859, Cyst: AUC 0.7615, F1 0.7607, Mass: AUC 0.6094, F1 0.6211, Tumor: AUC 0.7063, F1 0.7400, Attenuation: Accuracy 0.6827, F1 0.6167, and Size: MSE 0.9959 cm (Liang et al., 30 Jun 2025).

Stage 2 fine-tunes Qwen2.5-VL-7B-Instruct with LoRA using LLAMA-Factory. The model receives a structured feature template plus the CT slice image and learns the mapping

$\hat{y} = f_{\theta}(I, \mathbf{z}),$

where $I$ is the CT slice and $\mathbf{z}$ is the structured feature vector or template. Training uses LoRA rank = 8, learning rate = 1e-4, batch size = 4, gradient accumulation = 8, a cosine scheduler, 200 epochs, bf16 mixed precision, evaluation every 50 steps, and the same window width = 400, window level = 50, and 512 × 512 resizing. Across 5-fold cross-validation, report generation achieved BLEU-4 = 29.58, ROUGE-1 = 28.31, ROUGE-2 = 9.59, and ROUGE-L = 21.99 (Liang et al., 30 Jun 2025).

This two-stage design is echoed, with different architectural choices, by 3D CT report generators that make structure or region evidence explicit: CT2Rep introduced the first reported framework for automated radiology report generation from 3D medical imaging, specifically non-contrast chest CT volumes (Hamamci et al., 2024); Reg2RG proposed the first region-guided referring and grounding framework for CT report generation (Chen et al., 2024); CT-GRAPH organized organ and organ-system features into a hierarchical graph (Kalisch et al., 7 Aug 2025); MedRegion-CT used Region Representative ($R^2$) Token Pooling, pseudo-masks, and patient-specific attributions (Kyung et al., 29 Jun 2025); and Structure Observation Driven Image-Text Contrastive Learning for Computed Tomography Report Generation treated CTRG as a structure-centric vision-language learning problem with structure-specific visual queries (Liu et al., 5 Mar 2026).

4. Region, structure, and retrieval as Report2CT design patterns

Subsequent CT report-generation systems broadened the Report2CT-style design space by making anatomical structure, region grounding, or retrieved text explicit. Reg2RG uses anatomical masks from a universal segmentation module, local feature decoupling into texture and geometry, region-report alignment with shuffled local features, and LLaMA2-7B with LoRA; on RadGenome-ChestCT, it achieved Precision: 0.423, Recall: 0.181, and F1: 0.253 for clinical efficacy (Chen et al., 2024). CT-GRAPH builds a hierarchy of fine-level nodes, coarse anatomical-system nodes, and a global node from frozen pretrained 3D encoder features and anatomical masks; on CT-RATE, it reports CE F1 = 0.296, compared with CT2Rep w/ LLaMA: CE F1 = 0.214 and Reg2RG: CE F1 = 0.217, and the abstract highlights an absolute 7.9\% improvement in F1 (Kalisch et al., 7 Aug 2025).

MedRegion-CT is a region-focused MLLM using a frozen RAD-DINO ViT-B vision encoder, LLaMA3-8B, pseudo-masks from SAT, six predefined chest regions, and deterministic extraction of organ volume, lesion count, lesion diameter, and lesion location from masks. On RadGenome-Chest CT, it reports BLEU-4: 0.290, ROUGE-L: 0.375, METEOR: 0.494, CA-F1: 0.450, Green score: 0.278, and GPT-4 score: 48.837 (Kyung et al., 29 Jun 2025). AdaRAG-CT reframes report generation as a response to an embedding bottleneck in contrastive 3D CT encoders, noting that CT-CLIP avg pool has $\text{dim}_{90} = 2$ out of 512 and participation ratio $\text{PR} = 1.4$; it adds an adaptive retrieval channel with a learned [RAG] token and improves Clinical F1 on CT-RATE from 0.420 (CT-Agent) to 0.480 (Liang et al., 16 Mar 2026). Ker-VLJEPA-3B instead uses a frozen, self-supervised LeJEPA ViT-Large, zone-constrained cross-attention that compresses slice embeddings into 32 spatially grounded visual tokens, and a four-phase curriculum; on CT-RATE, it reports macro F1 = 0.429, reaching 0.448 with threshold optimization (Bumgardner et al., 24 Mar 2026).

These systems differ substantially in mechanism, but they share a common Report2CT-style premise: global volume embeddings alone are usually insufficient for fine-grained CT reporting. This suggests that Report2CT has evolved from direct vision-to-language mapping toward architectures that expose anatomical regions, structure-aware tokens, or supplementary textual evidence to the generator.

5. Evaluation and diagnostic fidelity

Evaluation has become a central problem for Report2CT research because conventional metrics are often too coarse for CT reports. CT-FineBench was proposed as a benchmark for fine-grained factual consistency of CT reports, built from CT-RATE and Merlin through a Question-Answering-based process that first identifies clinically critical attributes and then converts them into QA pairs (Yuan et al., 27 Apr 2026). The final benchmark contains more than 44k QA pairs derived from 6,646 reports, and the companion training corpus CT-FineData contains over 439k QA pairs. Its scoring rule is {0, 0.5, 1}, with synonym-aware exact match for categorical attributes and relative-error thresholds for numeric attributes. In human-correlation experiments, two medical experts rated sampled generated reports on a 10-point factual accuracy scale, and on CT-RATE CT-FineBench reached the highest Pearson correlation at 0.622 (Yuan et al., 27 Apr 2026).

A complementary benchmark addresses abdominal Report2CT-style generation from non-contrast CT. A Multi-Center Benchmark for Abdominal Disease Diagnosis and Report Generation from Non-Contrast CT curated a retrospective multi-center cohort of 1,254 patients from two tertiary institutions, with Center 1 (internal cohort): 1,085 patients/volumes and Center 2 (external validation cohort): 169 patients/volumes (Elbakry et al., 15 Jun 2026). The dataset supports multi-label disease classification across 53 abdominal pathologies spanning 15 organs, and free-text report generation from NCCT to contrast-enhanced report targets. Under this benchmark, the abstract reports an average multi-organ AUC of 69.1% on the internal cohort and 63.1% on the external cohort. For report generation, the paper explicitly prioritizes GREEN and RadGraph-XL over BLEU-4, METEOR, BERTScore, and ROUGE-L, arguing that standard text-overlap metrics are not enough in radiology (Elbakry et al., 15 Jun 2026).

For Report2CT systems, these developments mean that evaluation is increasingly moving from surface similarity to attribute-level factual consistency, omission detection, and clinically oriented entity-relation scoring. A plausible implication is that future Report2CT work will be judged less by fluency alone and more by whether it preserves lesion location, lesion size, margin, density, and type.

6. Report supervision, transfer, and interactive extensions

Report2CT has also expanded beyond direct report generation or report-conditioned synthesis. X2CT-CLIP uses a tri-modal latent space comprising CT volume $C$, CT report $R$, and chest radiography $X$, where the report functions as the semantic intermediary that transfers CT knowledge into a CXR encoder (You et al., 4 Mar 2025). CT-CLIP’s pretrained CT and report encoders are frozen, only the CXR encoder is trained, and the actual setting uses $\hat{y} = f_{\theta}(I, \mathbf{z}),$0, $\hat{y} = f_{\theta}(I, \mathbf{z}),$1, and $\hat{y} = f_{\theta}(I, \mathbf{z}),$2 in the tri-modal objective. On CT-RATE, RadChest-CT, and MIMIC-CT, the paper reports zero-shot AUC examples of 0.716, 0.645, and 0.567, respectively (You et al., 4 Mar 2025). Here the report is neither label text nor generated output; it is the alignment anchor.

Learning Segmentation from Radiology Reports moves further toward voxel-wise supervision from text. The method, R-Super, uses Llama 3.1 70B AWQ to extract tumor presence, location, diameters, and count from reports, achieving about 96% accuracy for tumor presence/location extraction, and converts those attributes into Volume Loss and Ball Loss for CT tumor segmentation (Bassi et al., 8 Jul 2025). The private dataset UCSF-Train contains 6,718 CT–Report pairs, combined with public CT-mask datasets from AbdomenAtlas 2.0. The abstract reports that F1 Score increased by up to 16% with respect to training with masks only (Bassi et al., 8 Jul 2025).

A different extension is interactive lesion morphology reporting. Interactive Segmentation and Report Generation for CT Images jointly generates segmentation masks and a structured report with five lesion morphology attributes—Shape, Invasion, Density, Heterogeneity, and Surface—from 3D CT medical images (Gu et al., 5 Mar 2025). On 1535 CT scans and masks across 15 lesion types, the full model reports a test set average accuracy of 0.759 for structured report generation and 0.794 DSC / 4.303 HD95 for segmentation, with gallstone and liver cyst treated as zero-shot test cases (Gu et al., 5 Mar 2025). Although this is not free-form report generation, it shows that Report2CT-style ideas can be instantiated as interactive, visually grounded structured reporting.

7. Limitations and open directions

The current Report2CT literature repeatedly identifies data scarcity, imbalance, and evaluation mismatch as limiting factors. In renal CT, the dataset is modest and highly imbalanced, many reports do not mention certain features, and the system uses one slice per sample, losing 3D lesion extent, cross-slice spatial continuity, and volumetric context; the paper therefore identifies extension to 3D CT volumes as the main future direction (Liang et al., 30 Jun 2025). In report-conditioned 3D CT synthesis, the method is computationally demanding, depends on detailed reports with findings and impression, and does not compare directly with Text2CT because of lack of public code or paired synthetic outputs (Amirrajab et al., 18 Sep 2025).

Generalization remains difficult. The multi-center abdominal NCCT benchmark reports substantial external drops and explicitly notes that enhancement-kinetic findings such as arterial hyperenhancement, portal venous washout, delayed retention, and capsule appearance cannot truly be recovered from NCCT, so an information ceiling remains (Elbakry et al., 15 Jun 2026). In retrieval-augmented report generation, AdaRAG-CT characterizes itself as a workaround rather than a root-cause fix, because the retrieval database is built entirely from the training corpus and the true limitation lies in impoverished visual representations (Liang et al., 16 Mar 2026). Evaluation is also unresolved: CT-FineBench is especially good at detecting omissions because questions are constructed from facts present in the reference report, but it is less suited to penalizing hallucinated findings not anchored in the reference (Yuan et al., 27 Apr 2026).

Across these studies, the dominant research directions are consistent: extension from 2D slices to 3D volumes, larger and better balanced datasets, domain-specific pretraining, more explicit grounding and localization, clinically grounded evaluation beyond BLEU and ROUGE, better external generalization across institutions, and tighter integration between report generation and CT generation. This suggests that Report2CT is evolving from a narrowly defined model name into a broader research program centered on clinically faithful report-conditioned and report-grounded CT intelligence.