MATNet: a multimodal attention-based temporal network integrating functional MRI and structural MRI for accurate ADHD classification

The complexity and heterogeneity of attention deficit hyperactivity disorder (ADHD) pose significant challenges for accurate diagnosis. Current studies primarily rely on unimodal magnetic resonance imaging (MRI) techniques, which fail to fully exploit the complementary characteristics of structural MRI (sMRI) and functional MRI (fMRI) data, thus limiting classification performance. To address this issue, we propose a innovative multimodal ADHD classification framework, termed multimodal attention-based temporal network (MATNet). MATNet constructs a graph structure based on the automated anatomical labeling brain region time series derived from fMRI and employs a spatiotemporal attention-based graph convolution module to extract dynamic spatiotemporal features. Simultaneously, an adaptive attention mechanism is integrated to focus on key brain regions relevant to the classification task. In addition, MATNet incorporates a 3D depthwise separable convolution and attention mechanism module to extract critical structural features from sMRI data. By employing a dynamic weighting strategy, MATNet achieves effective integration of multimodal features. Experimental results on the ADHD-200 dataset demonstrate that MATNet significantly enhances classification performance, achieving accuracies of 82.22%, 74.29%, and 85.49% on the Kennedy Krieger Institute, Neuroimaging International Center, and New York University Medical (NYU) Center sites, respectively. Compared with the second-best method (Hi-GCN), MATNet improves accuracy by 11.91% and F1 score by 11.01% on the NYU dataset, with an area under the curve of 0.89, highlighting its superior discriminative ability. We suggest that MATNet provides a reliable tool for precise ADHD diagnosis while opening new avenues for research and applications in multimodal neuroimaging.