A segmentation method with a large vision model for magnetic resonance imaging‐guided adaptive radiotherapy

BACKGROUND: Segmentation is the most effort-consuming step for magnetic resonance imaging guided adaptive radiotherapy (MRIgART). Although the segment anything model (SAM) exhibits impressive capabilities, its application in medical imaging necessitates clicks, bounding boxes, or providing mask prompts on each target image, which would still require complex human interactions. PURPOSE: This study introduces SAM-ART, a large vision model that integrates personalized information to enhance the segmentation accuracy of MRIgART. METHODS: This study utilized planning computed tomography (pCT), approved contours, and daily MRI (dMRI) from 38 patients with prostate cancer and 10 patients with rectal cancer. SAM-ART comprises an image encoder, a prompt encoder, and a mask decoder. Using mask and box prompts, SAM-ART propagates contours from pCT to dMRI using deformable image registration (DIR) and employs them as mask prompts, providing patient-specific information. The box prompts are used in slices prone to false negative (FN) predictions. A 5-fold cross-validation was then conducted, comparing SAM-ART with DIR, traditional deep learning (tDL), and SAM-ART using other manual prompts (point or box). RESULTS: The proposed SAM-ART exhibited a mean dice similarity coefficient of 0.934 ± 0.023 for the regions of interest, surpassing DIR (0.873 ± 0.063) and tDL (0.887 ± 0.056). Moreover, the proposed mask/box prompts also outperformed the other modes (point: 0.910 ± 0.027, and box: 0.921 ± 0.025). Mask/box prompts effectively mitigated FN predictions with minimal manual intervention. The ratio of acceptable slices (using the criteria of dice ≥ 0.85, 95th percentile of Hausdorff distance ≤ 5 mm, and mean distance to agreement ≤ 1.5 mm) was 89.38% with the proposed method, which means that the segmentations on about 90% of the slices did not require manual modification. CONCLUSIONS: This study proposed a novel method that integrates personalized information and manual prompts into a SAM-based segmentation model. It outperformed the baseline methods, with only a few contours needing to be revised for clinical use.