RoentMod: CXR Counterfactuals & Shortcut Correction

• RoentMod is a counterfactual image modification framework for chest radiographs that creates pathology-specific synthetic images while preserving original anatomy.
• It integrates a pretrained CXR generator and Stable Diffusion-based image editing to quantify and mitigate shortcut learning in diagnostic models.
• The framework enhances model generalization through targeted data augmentation, validated by radiologist studies and improved performance metrics.

RoentMod is a counterfactual image modification framework for chest radiographs (CXRs) designed to probe, quantify, and mitigate shortcut learning in deep learning-based medical image interpretation models. Shortcut learning refers to models exploiting spurious or off-target correlations rather than relying on clinicopathologically meaningful features, critically undermining the specificity and generalizability of AI systems in radiological decision-making. RoentMod generates anatomically realistic synthetic CXRs with user-specified pathology while preserving uninvolved anatomical features from the original image, enabling rigorous assessment and targeted correction of diagnostic model vulnerabilities.

1. Framework and Components

RoentMod integrates two principal architectural elements:

• RoentGen: An open-source, pretrained CXR generation model that is domain-optimized for chest radiograph synthesis and encoding of relevant anatomical and pathological priors.
• Image-to-Image Modification Using Stable Diffusion: RoentMod leverages the latent space editing capabilities of Stable Diffusion through a variational autoencoder and denoising U-Net architecture, adapted for medical image translation. Given a real CXR and a text prompt specifying desired pathology, the original image is projected into its latent representation, which is stochastically edited according to user input. Two tunable hyperparameters—guidance scale and modification strength—respectively control fidelity to the prompt and degree of morphological alteration.

The overall workflow is:

1. Input a CXR $I_0$ and a textual pathology description $T$.
2. Encode $I_0$ into latent space $z$.
3. Perform diffusion-based editing parameterized by $T$ and $(\text{guidance scale}, \text{strength})$.
4. Decode edited representation $z^*$ to synthetic image $I^*$.

This pipeline does not require additional fine-tuning and preserves subject-level anatomy outside the modified pathological region.

2. Model Validation and Performance Metrics

RoentMod's output was rigorously validated via radiologist reader studies and embedding-based similarity metrics:

• Realism and Adherence: In studies with board-certified radiologists and residents, 93% of RoentMod-generated CXRs were rated as anatomically realistic. Inclusion of the specified pathology matched radiologist assessment in 89–99% of cases, indicating robust conditioning.
• Subject Identity Preservation: Using pairwise Fréchet Inception Distance (pFID) across multiple embedding spaces (InceptionV3, XResNet, CLIP), synthetic images maintained anatomical similarity to the source CXR, comparable to real follow-up studies.
• Diagnostic Model Impact: Introduction of counterfactual pathology via RoentMod revealed shortcut learning in state-of-the-art multi-task and foundation models. Probability percentiles for non-prompted pathologies shifted substantially upon synthetic modification, substantiating reliance on off-target cues.

Table: Summary of Reader Study Outcomes

Metric	RoentMod Output	Range
Realistic Appearance	Radiologist validation	93%
Correct Pathology Addition	Radiologist validation	89–99%
Subject Identity Preservation	pFID (multiple embeddings)	Comparable to real follow-up

3. Correction of Shortcut Learning in Training

RoentMod enables controlled interventions for data augmentation and fine-grained discrimination:

• Training Augmentation: Counterfactual CXRs are introduced into the training set, forcing models to disambiguate true pathology from spurious features.
• Performance Gains: Incorporation of RoentMod images led to Area Under the ROC Curve (AUC) improvements of 3–19% on internal test sets (e.g., NIH CXR-14, MIMIC-CXR) and 1–11% on external cohorts (PadChest, CheXpert) for 5 out of 6 tested pathologies.
• Generalization: Augmented models showed enhanced generalization and decreased reliance on shortcut correlations, as measured by reduced prediction changes for non-targeted pathologies.

4. Counterfactual Generation: Methodological Considerations

Key methodological innovations:

• Text-to-image Conditioning: Modification is driven by explicit textual prompts corresponding to radiological findings, allowing precise, user-controlled intervention in image synthesis.
• Guidance Scale and Strength Tuning: Systematic variation of these parameters enables targeted alteration of pathological regions while minimizing collateral changes elsewhere in the anatomy.
• No Need for Retraining: RoentMod operates on pretrained models, facilitating rapid prototyping and stress-testing without computationally expensive retraining.

A plausible implication is that this architecture can be adapted to other imaging modalities that suffer from shortcut learning, provided domain-specific generative models and diffusion frameworks are available.

5. Applications Beyond Chest X-Ray Analysis

While developed for CXR, RoentMod's approach to counterfactual editing can extend to broader domains in medical AI:

• Fairness and Bias Assessment: Synthetic interventions can diagnose model bias with respect to demographic, technical, or acquisition variables.
• Stress Testing and Interpretability: By systematically manipulating image features, one can expose model decision boundaries and elucidate regions of high reliance on non-causal cues.
• Augmentation in Report Generation and Segmentation: Edited images supply realistic, pathology-specific exemplars for downstream tasks such as automated report writing or anatomical segmentation.
• Other Modalities: Extension to modalities such as MRI or CT is possible given analogous generative backbones and appropriate latent-editing frameworks.

6. Limitations, Conclusions, and Future Directions

RoentMod is currently limited to single-pathology editing per instance. The authors note future work may involve:

• Multi-pathology Modification: Extension to simultaneous editing of multiple findings in a single image.
• Region-Specific Causal Editing: Adoption of structural causal models for finer spatial specificity and localization of edits.
• Wider Dataset and Modality Coverage: Application to additional imaging cohorts and further validation in diverse clinical scenarios.

In conclusion, RoentMod advances the interpretability, robustness, and clinical reliability of deep learning models for chest radiograph interpretation. By enabling anatomically faithful, targeted counterfactual editing without retraining, it provides a practical and generalizable strategy to diagnose and correct shortcut learning, thereby improving the synergy between radiologists and AI-based diagnostic systems (Cooke et al., 10 Sep 2025).