计算机科学
图像融合
人工智能
融合
计算机视觉
分割
医学影像学
图像(数学)
一致性(知识库)
过程(计算)
图像分割
变压器
可视化
模式识别(心理学)
图像处理
特征提取
冗余(工程)
桥接(联网)
深度学习
数据挖掘
语义鸿沟
融合规则
传感器融合
模式
信息融合
作者
Yunzhan Fu,Xiangyu Shen,Yifei Sun,Yuhan Chen,Jian Wu,Hongxia Xu
出处
期刊:
[Association for Computing Machinery]
日期:2026-11-05
卷期号:: 2091-2100
被引量:1
标识
DOI:10.1145/3767308.3836491
摘要
Medical image fusion aims to integrate complementary information from diverse imaging modalities to support clinical diagnosis. Existing methods typically apply uniform fusion rules globally, lacking a deep understanding of diagnostic intents and pathological structures. To address these limitations, we propose MIND, a Multimodal Intent-Driven Network via Diffusion Transformers (DiTs) for medical image fusion. Specifically, we utilize BioMedGPT to generate intent-driven fusion texts from source images, guiding the fusion process with pathology-aware diagnostic intents. To combat the loss of 2D spatial continuity caused by 1D sequence flattening in DiTs, we design a Multi-scale Latent Adapter. This module explicitly extracts source image features before serialization, injecting them into the network via strict dimensional alignment to effectively supplement image features. To resolve the semantic shift caused by decoupling image outputs from diagnostic intents, we design a medical semantic consistency loss. This loss ensures deep semantic locking between fused images and fusion texts while maintaining the stability of the underlying physical manifold reconstruction. Comprehensive experiments on the Harvard, BraTS, and GFP datasets reveal that MIND delivers superior fusion quality, significantly improves downstream brain tumor segmentation accuracy, and enables flexible interactive fusion, holding significant promise for intent-driven intelligent clinical decision support systems.
科研通智能强力驱动
Strongly Powered by AbleSci AI