Customized Segment Anything Model for Medical Image Segmentation (2304.13785v2)

Abstract: We propose SAMed, a general solution for medical image segmentation. Different from the previous methods, SAMed is built upon the large-scale image segmentation model, Segment Anything Model (SAM), to explore the new research paradigm of customizing large-scale models for medical image segmentation. SAMed applies the low-rank-based (LoRA) finetuning strategy to the SAM image encoder and finetunes it together with the prompt encoder and the mask decoder on labeled medical image segmentation datasets. We also observe the warmup finetuning strategy and the AdamW optimizer lead SAMed to successful convergence and lower loss. Different from SAM, SAMed could perform semantic segmentation on medical images. Our trained SAMed model achieves 81.88 DSC and 20.64 HD on the Synapse multi-organ segmentation dataset, which is on par with the state-of-the-art methods. We conduct extensive experiments to validate the effectiveness of our design. Since SAMed only updates a small fraction of the SAM parameters, its deployment cost and storage cost are quite marginal in practical usage. The code of SAMed is available at https://github.com/hitachinsk/SAMed.

• The paper presents SAMed, which employs a low-rank adaptation strategy to tailor SAM for efficient medical image segmentation.
• It fine-tunes the image encoder, prompt encoder, and mask decoder with minimal parameter updates for cost-effective performance.
• Evaluated on the Synapse dataset, SAMed achieves a Dice Similarity Coefficient of 81.88 and a Hausdorff Distance of 20.64, rivaling state-of-the-art methods.

Customized Segment Anything Model for Medical Image Segmentation

The paper "Customized Segment Anything Model for Medical Image Segmentation" presents SAMed, a tailored approach to medical image segmentation by leveraging the capabilities of large-scale image segmentation models, specifically the Segment Anything Model (SAM). This work introduces a novel customization of SAM using a low-rank adaptation strategy, primarily focusing on enhancing efficiency and efficacy in medical contexts.

Methodological Approach

SAMed differentiates itself from traditional models by employing LoRA (Low-Rank Adaptation) for fine-tuning the SAM model specifically for medical image segmentation tasks. This entails modifying the image encoder, prompt encoder, and mask decoder with minimal parameter adjustment to maintain cost-efficiency. The deployment of LoRA enables SAMed to adapt the transformer layers within SAM, optimizing the model to handle the complexities inherent in medical images without significant overhead.

The adaptation strategy involves utilizing a minimal number of trainable parameters to achieve custom segmentation with semantic labels—a critical requirement in medical imaging that focuses on understanding anatomical structures. Importantly, SAMed employs a warmup fine-tuning strategy alongside the AdamW optimizer to stabilize and enhance the convergence of the model.

Numerical Results

Evaluated on the Synapse multi-organ segmentation dataset, SAMed achieves a Dice Similarity Coefficient (DSC) of 81.88 and a Hausdorff Distance (HD) of 20.64. These results demonstrate performance on par with state-of-the-art (SOTA) methods, particularly excelling in specific organ segmentation tasks such as pancreas and stomach. The method's ability to deliver competitive results against specialized models underscores the adaptability and capability of modified large-scale models in specialized domains.

Implications and Future Prospects

The implications of this research are significant for the field of medical imaging. By bridging large-scale models with domain-specific tasks, SAMed presents a scalable and economically feasible solution for medical image segmentation. By updating only a small fraction of parameters, SAMed significantly reduces deployment and storage costs, making it highly applicable in real-world settings.

Theoretically, this approach signifies a shift towards integrating large-scale models within specialized fields, challenging the necessity for entirely bespoke solutions. Methodologies like SAMed can potentially transform how medical imaging tasks are approached, offering robust solutions without exhaustive specialized engineering efforts.

Conclusion

SAMed exemplifies the successful customization of a large-scale segmentation model for medical image analysis, providing valuable insights into the intersection of general AI capabilities with domain-specific applications. Future research could extend this paradigm to other medical imaging modalities or even explore the integration of additional anatomical knowledge to further enrich the model's capabilities. Additionally, exploring the application of similar approaches to other fields could reveal broader implications of this methodology. This paper positions SAMed as an effective solution with substantial promise for advancing automated medical diagnostics.