Automatic cell segmentation using mini-u-net on fluorescence in situ hybridization images

Fluorescent in situ hybridization (FISH) is a molecular cytogenetic technique that provides reliable imaging biomarkers to diagnose cancer and genetic disorders in the cellular level. One prerequisite step to identify carcinoma cells in FISH images is to accurately segment cells, so as to quantify DNA/RNA signals within each cell. Manual cell segmentation is a tedious and time-consuming task, which demands automatic methods. However, automatic cell segmentation is hindered by low image contrast, weak cell boundaries, and cell touching in FISH images. In this paper, we develop a fast mini-U-Net method to address these challenges. Some special characteristics are tailored in the mini-U-Net, including connections between input images and their feature maps to accurately localize cells, mlpcon (multilayer perceptron + convolution) to segment cell regions, and morphology operators and the watershed algorithm to separate each individual cell. In comparison with the U-Net, the miniU-Net has fewer training parameters and less computational cost. The validation on 510 cells indicated that the Dice coefficients of the mini-U-Net and U-Net were 80.20% and 77.27%, and area overlap ratios were 69.17% and 68.04%, respectively. These promising results suggest that the mini-U-Net could generate accurate cell segmentation for fully automatic FISH image analysis.