Segment Anything in Medical Images

Jun Ma, Bo Wang

Development of MedSAM

The article discusses the development of MedSAM, a medical image segmentation tool based on the Segment Anything Model (SAM). Although successful in natural image segmentation, SAM's performance on medical images is limited. To address this, the authors curated a large-scale medical image dataset with over 200,000 masks across 11 modalities and developed a fine-tuning method, adapting SAM to general medical image segmentation. Comprehensive experiments on 21 3D and 9 2D segmentation tasks demonstrate MedSAM's superior performance to the default SAM model, with average Dice Similarity Coefficient (DSC) improvements of 22.5% and 17.6% on 3D and 2D tasks, respectively.

Importance of Accurate Medical Image Segmentation

Accurate segmentation of medical images is crucial for clinical applications such as disease diagnosis, treatment planning, and monitoring disease progression. Despite deep learning-based models showing promise in medical image segmentation, their generalization ability is limited. Therefore, developing foundation models adaptable to various medical imaging modalities and segmentation targets is essential in advancing medical image analysis.

MedSAM adapts SAM's transformer-based architecture to medical image segmentation by selecting the bounding box prompt and fine-tuning the mask decoder. The large-scale, diverse dataset includes 33 segmentation tasks, with MedSAM showing significant improvements in identifying challenging segmentation targets, such as small objects, weak boundaries, and interference from high-contrast objects. Although it achieves consistent improvements across various tasks and image modalities, its overall performance still falls behind specialist models specific to segmentation tasks. The authors suggest that larger models and increased dataset sizes could overcome MedSAM's limitations, and they plan to incorporate scribble-based prompts and integrate it into commonly used medical image viewers. By providing a step-by-step tutorial on fine-tuning SAM on new datasets, the authors aim to advance segmentation foundation models in the medical image domain. The code and trained model of MedSAM are publicly available, and the authors acknowledge dataset providers.

Link to Article: [2304.12306] Segment Anything in Medical Images (arxiv.org)