Pancreas segmentation by two-view feature learning and multi-scale supervision

• A two-view and multi-scale supervision network is proposed for pancreas segmentation. • A location branch for localization and a segmentation branch for segmentation. • Multi-scale supervision is used to learn multi-scale features. • Effectiveness verified on two pancreas datasets and one spleen dataset. Automatic organ segmentation systems can accelerate the development of computer-aided diagnosis (CAD) in clinical applications. In this paper, we focus on the challenging pancreas segmentation task. The tiny size, poor contrast, and blurred boundaries of the pancreas make it hard to detect. Current approaches emphasize decomposing this task into subtasks (localization and segmentation) and using the same network to solve different tasks. However, they overestimate the generalization ability of their models. In addition, current methods rely too much on the result of localization. To address these challenges, we propose a novel network by two-view feature learning based on attention mechanism and multi-scale supervision, which we term TVMS-Net. For localization, we adopt Attention Gate (AG) to distinguish appearance features of the pancreas in shallow layers. For segmentation, a simple and effective Residual Multi-Scale Dilated Attention (RMSA) module is designed to extract comprehensive inter-channel relationships and multi-scale spatial information. TVMS-Net is supervised in multi-scale to learn specific-level semantic information. Experimental results on two pancreas datasets show that TVMS-Net obtains remarkable performance. Importantly, TVMS-Net also achieves excellent segmentation accuracy on another tiny organ dataset, i.e., the spleen, which justifies the reliability and robustness of our method.