计算机科学
青光眼
人工智能
光学相干层析成像
医学影像学
模态(人机交互)
模式识别(心理学)
深度学习
图形
编码器
可视化
特征学习
计算机视觉
特征提取
自编码
眼底(子宫)
检眼镜
扫描激光检眼镜
图像融合
机器学习
计算机辅助诊断
判别式
连贯性(哲学赌博策略)
作者
Jiao Wang,Chi Liu,Y Zhang,Hongchen Luo,Zhifen Guo,Ying Hu,Ke Xu,Jing Zhou,Hongyan Xu,Ruiting Zhou,Man Tang
标识
DOI:10.1109/tmi.2026.3692814
摘要
Glaucoma is a leading cause of irreversible blindness worldwide, with asymptomatic early stages often delaying diagnosis and treatment. Early and accurate diagnosis requires integrating complementary information from multiple ocular imaging modalities. However, most existing studies rely on single- or dual-modality imaging, such as fundus and optical coherence tomography (OCT), for coarse binary classification, thereby restricting the exploitation of complementary information and hindering both early diagnosis and stage-specific treatment. To address these limitations, we propose glaucoma lesion evaluation and analysis with multimodal imaging (GLEAM), the first publicly available tri-modal glaucoma dataset comprising scanning laser ophthalmoscopy fundus images, circumpapillary OCT images, and visual field pattern deviation maps, annotated with four disease stages, enabling effective exploitation of multimodal complementary information and facilitating accurate diagnosis and treatment across disease stages. To effectively integrate cross-modal information, we propose hierarchical attentive masked modeling (HAMM) for multimodal glaucoma classification. Our framework employs hierarchical attentive encoders and light decoders to focus cross-modal representation learning on the encoder. The attention module, named multimodal-channel graph attention (MCGA), boosts glaucoma classification performance by emulating two key clinical reasoning steps: first, it uses a multi-head modality gating mechanism to replicate ophthalmologists' confidence scoring of fundus, OCT, and VF modalities; then, MCGA leverages a relational graph attention network to cross-examine structural-functional consistencies of weighted modalities. The experiments on GLEAM demonstrate that tri-modal fusion significantly outperforms single-modal and dual-modal configurations. Moreover, our proposed HAMM achieves superior performance compared with state-of-the-art multimodal learning methods. The dataset and code are publicly available via https://github.com/microewing/HAMM.
科研通智能强力驱动
Strongly Powered by AbleSci AI