Multimodal Masked Autoencoder Based on Adaptive Masking for Vitiligo Stage Classification

Vitiligo, a prevalent skin condition characterized by depigmentation, presents challenges in staging due to its inherent complexity. Multimodal skin images can provide complementary information, and in this study, the integration of clinical images of vitiligo and those obtained under Wood's lamp is conducive to the classification of vitiligo stages. However, difficulties in annotating multimodal data and the scarcity of multimodal data limit the performance of deep learning models in related classification tasks. To address these issues, a Multimodal Masked Autoencoder (Multi-MAE) based on adaptive masking is proposed in annotating multimodal data and the problem of multimodal data scarcity, and enhances the model's ability to extract characteristics from multimodal data. Specifically, an image reconstruction task is constructed to diminish reliance on annotated multimodal data, and a pre-training strategy is employed to alleviate the scarcity of multimodal data. Experimental results demonstrate that the proposed model achieves a vitiligo stage classification accuracy of 95.48% on a dataset of unlabeled dermatological images, an improvement of 5.16%, 4.51%, 3.87%, 2.58%, 4.51%, 4.51%, 3.87%, and 2.58% over that of MobileNet, DenseNet, VGG, ResNet-50, BEIT, MaskFeat, SimMIM, and MAE, respectively. These results verify the effectiveness of the proposed Multi-MAE model in assessing the stable and active vitiligo stages, making it a suitable clinical aid for evaluating the severity of vitiligo lesions.