ReXGroundingCT: 3D CT Free-Text Grounding

• The paper introduces ReXGroundingCT, the first public 3D chest CT dataset that links narrative radiology findings with detailed voxel-level segmentations.
• It leverages GPT-4 for standardizing reports and extracting findings, ensuring high annotation precision and robust quality control through manual radiologist supervision.
• The dataset supports tasks like sentence-level grounding, free-text medical segmentation, and grounded report generation focused on lung and pleural abnormalities.

ReXGroundingCT is a 3D chest CT dataset for segmentation of findings from free-text reports and is described as the first publicly available dataset to link free-text radiology findings with pixel-level segmentations in 3D chest CT scans that is manually annotated (Baharoon et al., 29 Jul 2025). It pairs standardized radiology report findings with spatially localized 3D segmentation annotations in volumetric imaging, with the stated aim of enabling sentence-level grounding, free-text medical segmentation, and grounded radiology report generation. Built on CT-RATE, it focuses on lung and pleural findings, preserves the descriptive content of clinical language rather than reducing reports to a fixed set of structured labels, and associates each finding with one or more 3D masks representing entities (Baharoon et al., 29 Jul 2025).

1. Conceptual scope and relation to earlier CT grounding work

ReXGroundingCT addresses a specific gap in chest CT: grounding free-text findings such as “3 mm nodule in the left lower lobe” to their precise 3D spatial extent in the CT volume. Its stated novelty lies in linking narrative report sentences to pixel-level volumetric segmentations, thereby supporting sentence-level grounding and free-text medical segmentation rather than only structured classification or category-constrained segmentation (Baharoon et al., 29 Jul 2025).

This design differs from prior CT resources that emphasize structured labels, coarse taxonomies, organ masks, or selected lesion types without linking free-text descriptors to 3D masks per sentence. The comparison in the dataset paper places ReXGroundingCT against structured-label CT datasets such as ULS23 universal lesion segmentation, LIDC-IDRI nodules, TotalSegmentator organs, and LiTS liver tumors, as well as against related grounded or multimodal resources such as PadChest-GR, OminiAbnorm-CT, RadGenome-Chest CT, RadGPT, and MedTrinity-25M; the distinguishing feature is manual disease-level sentence grounding with dense 3D masks for chest CT (Baharoon et al., 29 Jul 2025).

The immediate predecessor in CT visual grounding is the whole-report framework presented in “Visual Grounding of Whole Radiology Reports for 3D CT Images,” which claims the first visual grounding framework for 3D CT across various body parts and a wide variety of anomaly types, together with a large dataset containing 10,410 studies, 11,163 volumes, and 17,536 annotated regions (Ichinose et al., 2023). That earlier framework covered about 130 anatomy-by-anomaly combinations and used anatomical segmentation plus report structuring to localize anomalies from long, multi-anomaly reports, but no public release information was provided for its dataset, code, or models (Ichinose et al., 2023). ReXGroundingCT is narrower in anatomical scope—lung and pleural findings only—but public, manually annotated, and explicitly organized around sentence-level free-text grounding.

2. Dataset composition and representational scheme

ReXGroundingCT is built on CT-RATE, a large public dataset of non-contrast chest CT studies from Istanbul Medipol University acquired during 2015–2023, with English-translated, standardized reports. From CT-RATE, ReXGroundingCT curates 3,142 chest CT volumes and links standardized report findings to 3D masks (Baharoon et al., 29 Jul 2025).

Component	Value	Notes
CT scans	3,142	Non-contrast chest CT
Annotated findings	8,028	Lung and pleural findings
Segmented entities	16,301	One entity = one segmented instance
Training cases	2,992	Up to three representative segmentations per finding
Validation cases	50	Exhaustive labeling
Test cases	100	Exhaustive labeling by board-certified radiologists

The representational unit is the finding-to-entity mapping. A finding is a free-text report statement after standardization and extraction; an entity is one segmented instance of that finding. Thus, three separate nodules described in one finding correspond to three entities. Bilateral or multifocal findings are counted per instance. The training split caps annotations at three representative entities per finding, with an average of approximately 1.95 entities per finding, whereas validation and test use exhaustive enumeration, with an average of approximately 3.80 entities per finding across those splits (Baharoon et al., 29 Jul 2025).

The dataset includes both focal and non-focal abnormalities. Approximately 79% of findings are focal abnormalities and approximately 21% are non-focal. Focal examples include pulmonary nodules or masses, focal consolidation or atelectasis, ground-glass opacities, pleural effusion or thickening, honeycombing, and pneumothorax. Non-focal examples include emphysema, bronchiectasis, bronchial wall thickening, septal thickening or reticulation, and diffuse micronodules or tree-in-bud. Among focal findings, pulmonary nodules or masses are the most common category at 28.9% of focal findings; among non-focal findings, emphysema is the most common category at 27.7% of non-focal findings (Baharoon et al., 29 Jul 2025).

The report source schema retains the CT-RATE sections Clinical Info, Technique, Findings, and Impression. Reports were originally in Turkish, then machine-translated and corrected. ReXGroundingCT applies GPT-4 standardization to rewrite reports into US-style terminology and GPT-4 extraction to produce discrete findings. A hierarchical ontology with 12 parent categories and 61 subcategories is used for extraction, and segmentation results are summarized into 14 categories comprising 8 focal and 6 non-focal groups (Baharoon et al., 29 Jul 2025).

Acquisition characteristics reported for the curated cohort include a patient age of mean 42.08 ± 14.52 years, sex distribution of 55.9% male and 44.1% female, slice thickness of 0.50–5.00 mm, axial pixel spacing of 0.30–0.98 mm, and volumes containing 104–1005 slices with mean approximately 351. Matrix sizes are 512×512 in 92.6% of scans, 768×768 in 7.2%, and 1024×1024 in 0.2% (Baharoon et al., 29 Jul 2025).

3. Annotation pipeline, extraction logic, and quality control

The annotation workflow follows a three-stage pipeline. First, GPT-4 rewrites reports to standardize language to US radiology conventions, preserve clinical details, and remove irrelevant content. A board-certified radiologist reviewed 50 of 50 rewritten samples and found them acceptable (Baharoon et al., 29 Jul 2025).

Second, GPT-4 is used for finding extraction and GPT-4o-mini for categorization. The extraction rules split multi-finding sentences, consolidate multi-sentence descriptors, exclude interpretive or diagnostic language, and normalize measurements, converting millimeters to integers and centimeters to one decimal place. Manual review of 100 reports found no false positives; false negatives were reported as 0.05 at rephrasing and 0.01 at extraction; descriptor omissions were 0.14 at rephrasing and 0.13 at extraction. Categorization performance for lung and pleural findings was reported as precision 0.99 and recall 0.99 overall, with subcategory $F_1 = 0.92$ on a subset from lung and pleural categories (Baharoon et al., 29 Jul 2025).

Third, findings are manually segmented and then quality-controlled by radiologists. Training annotations follow two protocols: Protocol 1 covers 1,400 cases segmented by professional annotators and then refined or accepted by radiologists, whereas Protocol 2 covers 1,592 cases segmented by trained medical students under radiologist supervision with iterative correction. Validation and test annotations are produced exclusively by board-certified radiologists. The reported tooling includes brush, polygon, and region-growing operations, together with real-time slice navigation and window-level adjustment in a HIPAA-compliant platform (Baharoon et al., 29 Jul 2025).

Quality-control filtering is explicit. From 10,012 potential findings initially identified, 1,047 were excluded for being outside the lung and pleura scope, 449 were excluded because they were described but not visible, 367 were excluded because the diffuse patterns were not feasible to segment, and 121 were excluded as normal or benign variants. The remaining 8,028 findings were included for segmentation (Baharoon et al., 29 Jul 2025).

A common misunderstanding is to treat the training split as exhaustively labeled. The dataset paper states the opposite: training uses representative labeling with a cap of three entities per finding, while validation and test provide exhaustive labels. This suggests that training supervision is intentionally partial for multi-instance findings and that evaluation is stricter than the training annotation regime.

4. Supported tasks and evaluation protocols

The dataset is intended to support three tasks: sentence-level grounding, free-text medical segmentation, and grounded report generation. In sentence-level grounding, a model receives a free-text finding and is expected to localize and segment its 3D extent in the CT volume. In free-text medical segmentation, the input consists of the CT and free-text descriptions and the output is a per-finding mask. In grounded report generation, the goal is to generate findings together with spatial masks aligned to the report narrative (Baharoon et al., 29 Jul 2025).

The dataset paper introduces these tasks but does not stipulate an official metric suite or thresholds. It instead lists standard metrics appropriate for ReXGroundingCT. These include the Dice coefficient,

$$\mathrm{Dice}(A,B) = \frac{2\lvert A \cap B\rvert}{\lvert A\rvert + \lvert B\rvert},$$

the Intersection over Union,

$$\mathrm{IoU}(A,B) = \frac{\lvert A \cap B\rvert}{\lvert A \cup B\rvert},$$

the Hausdorff distance,

$$H(A,B) = \max\left\{\sup_{a\in A}\inf_{b\in B} d(a,b),\ \sup_{b\in B}\inf_{a\in A} d(a,b)\right\},$$

and the Average Symmetric Surface Distance,

$$\mathrm{ASSD}(A,B) = \frac{1}{\lvert S(A)\rvert + \lvert S(B)\rvert}\left(\sum_{a\in S(A)}\min_{b\in S(B)} d(a,b) + \sum_{b\in S(B)}\min_{a\in S(A)} d(b,a)\right).$$

For grounding, the paper also describes localization accuracy at an IoU threshold $\tau$, Hit@k in a retrieval setting, and Average Precision over IoU thresholds (Baharoon et al., 29 Jul 2025).

The sentence-to-mask mapping protocol is likewise specified at a methodological level rather than as an official leaderboard rule set. Each finding sentence is mapped to one or more entities. For evaluation, predicted entities may be matched to ground-truth entities of the same finding, for example via Hungarian matching on $1-\mathrm{IoU}$ costs. Partial matches may then be summarized by reporting per-entity IoU or Dice and aggregating by mean per finding or by micro and macro averages across the dataset. Multi-sentence descriptors are consolidated during extraction, so the grounding target is the unified finding text rather than the original sentence boundaries (Baharoon et al., 29 Jul 2025).

An important implication is that ReXGroundingCT defines the data substrate and tasks more explicitly than it defines a single canonical benchmark protocol. The paper itself notes future work to formalize evaluation metrics and leaderboards and to add baseline models (Baharoon et al., 29 Jul 2025).

5. Benchmark role and subsequent model development

ReXGroundingCT functions as a benchmark for later 3D chest CT grounding and text-conditioned segmentation models. In “EXACT: an explainable anomaly-aware vision foundation model for analysis of 3D chest CT,” the dataset is used for zero-shot anomaly localization and for few-shot supervised localization. EXACT reports internal ReX-Train/ReX-Val zero-shot anomaly localization with DSC 0.050/0.071 versus BiomedParse-v2 0.012/0.065, and AUPR 0.044/0.065 versus 0.026/0.028, together with higher Hit Rates. In the supervised few-shot setting, EXACT-Seg reports ReX-Val DSC 0.215 versus SegMamba 0.198 and AUPR 0.200 versus 0.187 (Bai et al., 27 Apr 2026).

The architectural reason given for this fit is that EXACT produces organ-constrained, disease-specific anomaly-aware maps at the voxel level. For ReXGroundingCT, the paper states that AAmap channels for lung and pleural pathologies such as ground-glass opacity, consolidation, atelectasis, pleural effusion, interlobular septal thickening, bronchiectasis, peribronchial thickening, pulmonary fibrotic sequela, and mosaic attenuation align well with ReX target entities (Bai et al., 27 Apr 2026). This suggests a close methodological correspondence between the dataset’s sentence-level lesion masks and foundation models that preserve voxel-level spatial evidence rather than compressing scans into global image-text embeddings.

ReXGroundingCT is also an explicit evaluation target for “VoxTell: Free-Text Promptable Universal 3D Medical Image Segmentation.” Following the official protocol, VoxTell was fine-tuned on the ReXGroundingCT training split, augmented with an instance-focused dataset curated by the authors, and outperformed SAT on the validation set: Dice 28.2 versus 13.1 and HIT5% 67.8% versus 49.8% (Rokuss et al., 14 Nov 2025). VoxTell attributes this performance to multi-stage text-image fusion across decoder layers, which is said to better preserve laterality and regional qualifiers such as “right upper lobe” throughout the feature hierarchy (Rokuss et al., 14 Nov 2025).

These follow-up results clarify ReXGroundingCT’s role in the literature. It is not only a release of annotated chest CT data; it is also a benchmark setting for open-vocabulary, sentence-conditioned volumetric localization, with models ranging from anatomy-aware weakly supervised foundation models to fully text-prompted segmentation systems.

6. Limitations, biases, access, and future directions

ReXGroundingCT has several explicit limitations. Its scope is restricted to lung and pleural findings, and non-pulmonary findings were excluded during quality control. The training split mixes professional annotators and medical students, albeit under radiologist supervision, while validation and test are annotated exclusively by board-certified radiologists. Training labels are partial because only up to three representative entities are segmented per finding. GPT-4 standardization and extraction introduce a possible source of semantic drift, even though reported error rates were low. The source is a single institution, which may limit generalization across vendors and acquisition protocols (Baharoon et al., 29 Jul 2025).

The dataset paper also notes that imaging preprocessing is not prescribed by the dataset. Reports were standardized for clarity and consistency, but annotators adjusted window levels during labeling and exact file formats are not specified in the paper. License and permitted uses are delegated to the Hugging Face dataset page, as are storage format, file organization, and any evaluation scripts (Baharoon et al., 29 Jul 2025).

Access is provided at https://huggingface.co/datasets/rajpurkarlab/ReXGroundingCT. The underlying CT-RATE imaging and reports underwent extensive anonymization, and the annotation platform is described as HIPAA-compliant. The paper states that the dataset contains no PHI according to the anonymization procedures described, while IRB specifics are not detailed (Baharoon et al., 29 Jul 2025).

The future directions listed for the resource are concrete: extension beyond lung and pleura to mediastinal, cardiovascular, and abdominal structures visible on chest CT; inclusion of contrast-enhanced studies; temporal grounding across prior and follow-up scans; formalization of evaluation metrics and leaderboards; addition of baseline models; and cross-institution validation and robustness studies (Baharoon et al., 29 Jul 2025). A plausible implication is that ReXGroundingCT currently serves as a focused but incomplete substrate for thoracic report grounding: rich enough to support sentence-level 3D grounding research, but intentionally limited in anatomical scope and evaluation standardization.