计算机科学
分割
人工智能
编码器
情态动词
模式识别(心理学)
模态(人机交互)
图像分割
深度学习
化学
高分子化学
操作系统
作者
Tianyu Shi,Huiyan Jiang,Bin Zheng
出处
期刊:IEEE Transactions on Biomedical Engineering
[Institute of Electrical and Electronics Engineers]
日期:2022-01-01
卷期号:69 (1): 108-118
被引量:22
标识
DOI:10.1109/tbme.2021.3087612
摘要
Based on the hypothesis that adding a cross-modal and cross-attention (C2MA) mechanism into a deep learning network improves accuracy and efficacy of medical image segmentation, we propose to test a novel network to segment acute ischemic stroke (AIS) lesions from four CT perfusion (CTP) maps.The proposed network uses a C2MA module directly to establish a spatial-wise relationship by using the multigroup non-local attention operation between two modal features and performs dynamic group-wise recalibration through group attention block. This C2MA-Net has a multipath encoder-decoder architecture, in which each modality is processed in different streams on the encoding path, and the pair related parameter modalities are used to bridge attention across multimodal information through the C2MA module. A public dataset involving 94 training and 62 test cases are used to build and evaluate the C2MA-Net. AIS segmentation results on testing cases are analyzed and compared with other state-of-the-art models reported in the literature.By calculating several average evaluation scores, C2MA-network improves Recall and F2 scores by 6% and 1%, respectively. In the ablation experiment, the F1 score of C2MA-Net is at least 7.8% higher than that of single-input single-modal self-attention networks.This study demonstrates advantages of applying C2MA-network to segment AIS lesions, which yields promising segmentation accuracy, and achieves semantic decoupling by processing different parameter modalities separately.Proving the potential of cross-modal interactions in attention to assist identifying new imaging biomarkers for more accurately predicting AIS prognosis in future studies.
科研通智能强力驱动
Strongly Powered by AbleSci AI