计算机科学
卷积神经网络
变压器
人工智能
融合
深度学习
领域(数学分析)
生成对抗网络
生成语法
模式识别(心理学)
数学分析
语言学
哲学
物理
数学
量子力学
电压
作者
Jie Zhang,Bicao Li,Bei Wang,Zhuhong Shao,Jie Huang,Ji-Qing Lu
标识
DOI:10.1007/978-981-99-8558-6_36
摘要
The existing methodologies for image fusion in deep learning predominantly focus on convolutional neural networks (CNN). However, recent investigations have explored the utilization of Transformer models to enhance fusion performance. Hence, we propose a novel generative adversarial fusion framework that combines both CNN and Transformer, referred to as the Cross-Domain Bidirectional Interaction Fusion Network (CDBIFusion). Specifically, we devise three distinct pathways for the generator, each serving a unique purpose. The two CNN pathways are employed to capture local information from MRI and PET images, and the other pathway adopts a transformer architecture that cascades both source images as input, enabling the exploitation of global correlations. Moreover, we present a cross-domain bidirectional interaction (CDBI) module that facilitates the retention and interaction of information deactivated by the ReLU activation function between two CNN paths. The interaction operates by cross-cascading ReLU activation features and deactivation features from separate paths by two ReLU rectifiers and then delivering them to the other path, thus reducing the valuable information lost through deactivation. Extensive experiments have demonstrated that our CDBIFusion surpasses other current methods in terms of subjective perception and objective evaluation.
科研通智能强力驱动
Strongly Powered by AbleSci AI