MRGen: Segmentation Data Engine For Underrepresented MRI Modalities (2412.04106v2)

Abstract: Training medical image segmentation models for rare yet clinically significant imaging modalities is challenging due to the scarcity of annotated data, and manual mask annotations can be costly and labor-intensive to acquire. This paper investigates leveraging generative models to synthesize training data, to train segmentation models for underrepresented modalities, particularly on annotation-scarce MRI. Concretely, our contributions are threefold: (i) we introduce MRGen-DB, a large-scale radiology image-text dataset comprising extensive samples with rich metadata, including modality labels, attributes, regions, and organs information, with a subset having pixelwise mask annotations; (ii) we present MRGen, a diffusion-based data engine for controllable medical image synthesis, conditioned on text prompts and segmentation masks. MRGen can generate realistic images for diverse MRI modalities lacking mask annotations, facilitating segmentation training in low-source domains; (iii) extensive experiments across multiple modalities demonstrate that MRGen significantly improves segmentation performance on unannotated modalities by providing high-quality synthetic data. We believe that our method bridges a critical gap in medical image analysis, extending segmentation capabilities to scenarios that are challenging to acquire manual annotations.

• The paper introduces MRGen, a diffusion-based controllable data engine addressing heterogeneous modalities and annotation scarcity in MRI segmentation.
• Key contributions include the large-scale MedGen-1M image-text dataset and a novel diffusion model generating training data conditioned on text prompts and segmentation masks without requiring paired data.
• Evaluation demonstrates that MRGen significantly boosts segmentation performance on target unannotated modalities compared to conventional methods, reducing reliance on manual annotations.

Diffusion-Based Controllable Data Engine for MRI Segmentation

The manuscript "MRGen: Diffusion-based Controllable Data Engine for MRI Segmentation towards Unannotated Modalities" presents a sophisticated approach to tackle the pressing issues of heterogeneous modalities and annotation scarcity in medical image segmentation. The authors propose MRGen, a diffusion-based data engine, providing a mechanism for controllable data synthesis which circumvents the need for registered data pairs prevalent in conventional methods.

Key Contributions and Methodology

The research primarily focuses on three novel contributions:

1. Dataset Curation: The authors introduce MedGen-1M, a comprehensive dataset consisting of radiology image-text pairs enriched with modality labels, attributes, region, and organ information. The dataset is instrumental in training controllable generative models intended for medical imaging applications, especially across unannotated modalities. This large-scale collection is a foundational element for subsequent model training and evaluation.
2. Innovative Data Engine: The diffusion-based model within MRGen is designed to generate MR images conditioned on both text prompts and segmentation masks. This model synthesizes training data for modalities lacking traditional segmentation annotations, thereby expanding the applicability of segmentation models across diverse imaging settings. The controllable aspect of MRGen is achieved through a two-stage training process and a specifically designed mask condition controller, enabling precise image generation aligned with given conditions without needing paired data.
3. Comprehensive Evaluation: The paper offers extensive quantitative and qualitative evaluations demonstrating that MRGen significantly boosts segmentation performance on target modalities, often unannotated, compared to existing data augmentation and translation techniques like CycleGAN and DualNorm.

Highlights of Strong Results

The evaluation results show that MRGen achieves remarkable performance improvements in segmentation tasks, underscoring its ability to generate high-fidelity and modality-accurate images. The models trained using MRGen-generated datasets noticeably outperform those relying on conventional augmentation methods when applied to unannotated modalities. The pre-eminence in achieving the lowest Frechet Inception Distance (FID) scores across various settings highlights its superior capability in image generation fidelity and diversity.

Theoretical and Practical Implications

The work presents both theoretical and practical implications:

• Theoretical Implications: The integration of diffusion models with controllable generation capabilities reflects a significant stride in adapting generative models for medical imaging. The methodology provides insights into overcoming data scarcity and heterogeneity challenges intrinsic in medical imaging, opening pathways for more robust, generalizable models.
• Practical Implications: On practical grounds, MRGen narrows the gap between annotated and unannotated MRI modalities, effectively extending the reach of MRI-based diagnostic tools. This could potentially reduce reliance on costly and time-consuming manual annotations.

Future Directions

Notwithstanding its impressive achievements, MRGen's performance can be further enhanced. Addressing limitations related to small organ mask conditions and false-negative sample generation can refine model accuracy. Exploring architectures with greater precision in generating low-volume organ representations will be beneficial. The prospect of adapting the framework to other imaging modalities like CT offers valuable avenues for future investigation.

In summary, the introduction of MRGen establishes a substantial leap in the domain of MRI segmentation, offering a scalable, efficient, and adaptable solution to the long-standing challenge of unannotated modality segmentation. This research lays a robust foundation for future exploration at the intersection of medical imaging and advanced generative models, with profound implications for enhancing automated medical diagnostic tools.