Interpretable learning approaches in structural MRI: 3D-ResNet fused attention for autism spectrum disorder classification

Are there any abnormal reflection in the structural Magnetic Resonance Imaging(sMRI) of patients with autism spectrum disorder (ASD)? Although a few brain regions have been somehow implicated in the pathophysiologic mechanism of the disorder, the gold-standard for diagnosis based on sMRI has not been reached in the academic community. Recently, the powerful deep learning algorithms have been widely studied and applied, which provides a chance to explore the brain structural abnormalities of ASD by the visualization based on the deep learning model. In this paper, a 3D-ResNet with an attention subnet for ASD classification is proposed. The model combined the residual module and the attention subnet to mask the regions which are relevant or irrelevant to the classification during the feature extraction. The model was trained and tested by sMRI from Autism Brain Imaging Data Exchange (ABIDE). The result of 5-fold cross-validation shows an accuracy of 75%. The Grad-CAM was further applied to display the emphasized composition of the model during classification. The class activation mapping of multiple slices of the representation sMRI was visualized. The results show that there are high related signals in the regions near the hippocampus, corpus callosum, thalamus, and amygdala. This result may confirm some of the previous hypotheses. The work is not only limited to the classification of ASD but also attempts to explore the anatomic abnormality with a quite promising visualization-based deep learning approach.