Attention augmented distance regression and classification network for nuclei instance segmentation and type classification in histology images

Nuclei instance segmentation and classification in histology plays a major role in routine pathology image examination, which enable morphological features analysis that further facilitates streamlined diagnosis and prognosis quantification. However, the nuclei in the tissue images obtained from different human organs are characterized with high variability in shape, size, morphology and spatial arrangements. Moreover, during digitization of tissue slide, the image quality is degraded because of added artifacts, poor contrast, blurred regions due to failed auto-focus and inconsistent staining procedure. Owing to these challenges, it is difficult to build a generalized feature representation that can achieve precise segmentation and classification of nuclei instances in complex tumor micro-environment of tissue specimens obtained from various organs. To address these problems, we propose a novel deep learning model, that harnesses horizontal and vertical distance information hidden among the nuclei instances to successfully delineate the challenging nuclei. Our proposed methodology uses soft attention mechanism to generate relevant feature activation and prune irrelevant and noisy information. These attention units produce more precise and refined feature maps resulting in finer instances segmentation and accurate classification in the overlapping nuclei, the nuclei with touching boundaries and reduction in false positives. We train our model on publicly available data-sets (Kumar, CoNSep, CPM-17 and a new data-set PanNuke). Our methodology shows superior performance in nuclei classification and segmentation in comparison with recently published methods. The code and the obtained results have been made public at the following link: http://t.ly/eGkV.