Multimodal Enhanced Learning for Skin Disease Image Analysis and Classification
作者
Mansi Wanjale,Sandhya Arora
标识
DOI:10.1109/i3ctcon68242.2026.11507229
摘要
In modern skin lesion classification models, effective diagnosis is necessary, but it is challenging due to visual ambiguity, lesion variability, and inter-observer inconsistency. This work proposes a multimodal deep learning model that integrates dermoscopic images with metadata and clinical notes about patients to improve diagnostic performance across four major skin lesion types: melanoma (mel), melanocytic nevus (nv), basal cell carcinoma (bcc), and benign keratosis (bkl). Datasets like HAM10000 and Derm7pt demonstrate that multimodal fusion enhances classification robustness, particularly for underrepresented classes. EfficientNetB3 is utilized as the extractor of visual features owing to its compound scaling and superior parameter efficiency compared to its predecessors. This study achieves comparative accuracy while considering class imbalance of the dataset by applying under-sampling and class-weighted loss functions.