Early diagnosis model of mycosis fungoides and five inflammatory skin diseases based on multi-modal data-based convolutional neural network

Abstract Background Mycosis fungoides (MF) is the most common type of cutaneous T-cell lymphoma, and early-stage MF is difficult to differentiate from erythematous inflammatory disease. Except biopsy, non-invasive information such as patient’s basic information, clinical images and dermoscopic images is of great significance for early diagnosis of MF. However, there is still a lack of diagnosis models based on convolutional neural network that can utilize the above multimodal information. Objectives We aim to develop an artificial intelligence (AI) deep learning model based on multimodal information, verify its classification efficiency, and construct an AI-aided early diagnostic model of MF and inflammatory skin diseases for dermatologists. Methods This is a single center retrospective study based on multimodal information including clinical information, clinical images, and dermoscopic images. A total of 1157 cases of MF and inflammatory diseases were collected, including 2452 clinical images, 6550 dermoscopic images and corresponding clinical data. RegNetY-400MF was selected as the feature extractors in the study. Results AI model demonstrates higher levels of total accuracy, precision, sensitivity, and specificity in classification of MF and other inflammatory skin diseases compared to the participating dermatologists. A significant enhancement was noticed in average accuracy, sensitivity, and specificity for MF and inflammatory diseases within the Doctor+AI group, with values of 82.94%, 86.16%, and 96.45% respectively, compared to 71.52%, 74.56%, and 94.06% within the Doctor-only group. The more accurately diagnosis of each disease was also achieved by the multi-classification model. Conclusions These results indicate that our AI model has a significantly strong discriminative ability to assist doctors in improving diagnostic accuracy of early-stage MF and common inflammatory skin diseases.