Multi-modal Adaptive Synchronous Learning for Imbalanced Dermatosis Diagnosis
作者
Ziyu Dong,Shibo Zhang,Xiaodong Yang,Xin Qin
出处
期刊:Conference proceedings [Institute of Electrical and Electronics Engineers] 日期:2025-10-05卷期号:: 7511-7516
标识
DOI:10.1109/smc58881.2025.11343474
摘要
Skin cancer is one of the most prevalent cancers worldwide. Early diagnosis of skin lesions is essential for improving survival rates. Multimodal learning with skin lesion images and clinical metadata has shown promise in boosting diagnostic performance. However, imbalanced multimodal learning remains a major challenge. The image modality, containing more decisive information, often dominates the learning process, suppressing effective learning from the metadata. Existing methods often overlook the inherent disparity in decisive information between modalities and tend to overemphasize less informative ones, leading to suboptimal performance. Moreover, the widespread use of one-hot encoding for categorical metadata creates sparse representations, which complicate weight updates in early network layers and further suppress metadata learning. To address these issues, we propose a Multimodal Adaptive Synchronous Learning (MASL) method. Specifically, we replace conventional one-hot encoding with a Dense Token Embedding (DTE) derived from a learnable embedding matrix. In addition, we introduce a Prior-Guided Adaptive Parameter Freezing (PGAPF) mechanism, which leverages the prior decision confidence ratio between modalities to adjust the multimodal learning process. Experiments on PAD-UFES-20 and ISIC 2019 demonstrate that MASL improves diagnostic performance and promotes more balanced multimodal learning, achieving 1–2% gains across key evaluation metrics, including AUC, accuracy, sensitivity, and specificity.