DETisSeg: A dual-encoder network for tissue semantic segmentation of histopathology image

Accurate segmentation of tissue regions in hematoxylin and eosin (H&E)-stained histopathology images is an essential step in computational pathology. It is still a challenging task due to the large-scale distribution, irregular morphological variations, and fuzzy boundaries between different tissues. Semantic segmentation methods with strong feature representative capability for extracting local fine-grained and global coarse-grained features are desired to tackle this challenge. In this paper, we propose a dual encoder network for tissue semantic segmentation of histopathology image, called DETisSeg, which aims to fully fuse the features of global context information by the Swin Transformer branch and local features by the convolutional neural network(CNN) branch. Particularly, we design an improved residual connection to recover the lost spatial information during the patch merging phase in the Swin Transformer branch. In addition, we employ a pyramid architecture decoder to generate a composite feature map with multiple scales. We perform experiments on two publicly available tissue semantic segmentation datasets, including the BCSS and LAUD-HistoSeg datasets, to evaluate the effectiveness of the proposed DETisSeg. Compared with eight state-of-the-art semantic segmentation methods, the results show that the proposed DETisSeg achieves improved performance. Particularly, it outperforms Swin transformer on the mean Dice and IoU with improvements of 1.36% and 1.79% on BCSS dataset, 0.96% and 4.48% on LUAD-HistoSeg dataset, respectively.