人工智能
模式
计算机科学
背景(考古学)
自编码
缺少数据
正交性
幻觉
机器学习
集合(抽象数据类型)
特征(语言学)
自然语言处理
医学影像学
统一医学语言系统
模式识别(心理学)
传感器融合
语义学(计算机科学)
数据挖掘
数据集
计算机视觉
自然语言
作者
Nagur Shareef Shaik,Teja Krishna Cherukuri,Adnan Masood,Dong Hye Ye
标识
DOI:10.1609/aaai.v40i11.37835
摘要
The integration of medical images with clinical context is essential for generating accurate and clinically interpretable radiology reports. However, current automated methods often rely on resource-heavy Large Language Models (LLMs) or static knowledge graphs and struggle with two fundamental challenges in real-world clinical data: (1) missing modalities, such as incomplete clinical context , and (2) feature entanglement, where mixed modality-specific and shared information leads to suboptimal fusion and clinically unfaithful hallucinated findings. To address these challenges, we propose the DiA-gnostic VLVAE, which achieves robust radiology reporting through Disentangled Alignment. Our framework is designed to be resilient to missing modalities by disentangling shared and modality-specific features using a Mixture-of-Experts (MoE) based Vision-Language Variational Autoencoder (VLVAE). A constrained optimization objective enforces orthogonality and alignment between these latent representations to prevent suboptimal fusion. A compact LLaMA-X decoder then uses these disentangled representations to generate reports efficiently. On the IU X-Ray and MIMIC-CXR datasets, DiA has set new state-of-the-art BLEU@4 scores of 0.266 and 0.134, respectively. Experimental results show that the proposed method significantly outperforms state-of-the-art models.
科研通智能强力驱动
Strongly Powered by AbleSci AI