Rectal cancer: Toward fully automatic discrimination of T2 and T3 rectal cancers using deep convolutional neural network

Abstract Preoperative chemoradiotherapy is known to reduce the local recurrence of locally advanced rectal cancer. However, the careful use of preoperative chemoradiotherapy is essential, because unnecessary over‐treatment can result in unintended complications. Therefore, a diagnostic system for distinguishing between T2 and T3 rectal cancers should be developed. According to the diagnostic criteria for rectal cancer, radiologists first identify the locations and the shapes of both the tumor and the rectum from a medical image and then diagnose the T2/T3 rectal cancer by determining whether the tumor passes through the rectal wall or not. We construct two distinct convolutional neural network models to achieve the automated segmentation of each rectum and tumor, respectively. Then, we construct another convolutional neural network model, which uses the output images of segmentation models as input and determines whether the tumor in the input magnetic resonance image is at the T2 stage or at the T3 stage. We evaluate the effectiveness of the proposed method based on 290 magnetic resonance images from 133 subjects. The proposed model demonstrates an accuracy of 94%.