Visual evidence for interpreting diagnostic decision of deep neural network in computer-aided diagnosis

Recent studies have reported that deep learning techniques could achieve high performance in medical image analysis such as computer-aided diagnosis (CADx). However, there is a limitation in interpreting the diagnostic decisions of deep learning due to the black-box nature. To increase confidence in the diagnostic decisions of deep learning, it is necessary to develop a deep neural network with the interpretable structure which could provide a reasonable explanation of diagnostic decisions. In this study, a novel deep neural network has been devised to provide visual evidence of the diagnostic decisions of CADx. The proposed deep network is designed to include a visual interpreter which could provide important areas as the visual evidence of the diagnostic decision in the deep neural network. Based on the observation that the radiologists usually make a diagnostic decision based on the lesion characteristics (the margin and the shape of masses), the visual interpreter provides visual evidence related with the margin and the shape, respectively. To verify the effectiveness of the proposed method, experiments were conducted on mammogram datasets. Experimental results show that the proposed method could provide more important areas as the visual evidence compared with the conventional visualization method. These results imply that the proposed visual interpretation method could be a promising approach to overcome the current limitation of the deep learning for CADx.