人工智能
计算机科学
嵌入
图像分割
计算机视觉
分割
图像(数学)
模式识别(心理学)
尺度空间分割
图像处理
蒸馏
医学影像学
特征提取
图像纹理
基于知识的系统
基于分割的对象分类
上下文图像分类
训练集
作者
Jiyang Tang,Hu Han,Shiguang Shan,Xilin Chen
标识
DOI:10.1109/tmi.2026.3674509
摘要
Surgical image segmentation is vital for various stages of surgical procedures, from preoperative planning to real-time navigation and postoperative assessment. Despite advances in deep learning, current surgical image segmentation methods remain limited. They primarily target instrument segmentation and show poor generalizability across different surgical settings. While the Segment Anything Model (SAM) shows robust generalization capabilities in the segmentation of natural images, adapting SAM to surgical and medical images faces challenges because of its reliance on high-quality user-provided prompts and inherent lack of design for multi-class semantic segmentation. To address these limitations, we propose Distillation-SAM, an effective method that adapts SAM for accurate surgical image segmentation without user-provided prompts while freezing its encoder and decoder. Distillation-SAM introduces a trainable adapter branch that learns both sparse auto-prompt embeddings and enriched image features with dense auto-prompt embeddings, enabling the segmentation of surgical objects such as vessels, instruments, and tissues. We propose a direct knowledge distillation constraint for these auto-prompt embedding learnings by using embeddings derived from ground-truth masks as guidance. To enable multi-class semantic segmentation using SAM, we revise the mask score regression branch in SAM's decoder by incorporating a trainable Multilayer Perceptron to predict mask categories while keeping other parameters frozen. Our experiments in multiple surgical datasets, including IVIS, EndoVis2017, and Cholecseg8k, demonstrate that distillation-SAM outperforms existing methods in vessel, tissue, and instrument segmentation.
科研通智能强力驱动
Strongly Powered by AbleSci AI