Early diagnosis model of mycosis fungoides and five inflammatory skin diseases based on a multimodal data-based convolutional neural network

Abstract Background Mycosis fungoides (MF) is the most common type of cutaneous T-cell lymphoma, and early-stage MF is difficult to differentiate from erythematous inflammatory disease. With the exception of biopsy, noninvasive information such as a patient’s medical history and clinical and dermoscopic images is of great significance for early diagnosis of MF. However, there is a lack of diagnostic models based on convolutional neural networks that can use multimodal information. Objectives To develop an artificial intelligence (AI) deep learning model based on multimodal information, to verify its classification efficiency and to construct an AI-aided early diagnostic model of MF and inflammatory skin diseases for dermatologists. Methods This was a single-centre retrospective study based on multimodal information, including clinical information, clinical images and dermoscopic images. A total of 1157 cases of MF and inflammatory diseases were collected, including 2452 clinical images, 6550 dermoscopic images and corresponding clinical data. To assess the practicality of using AI models to help with clinical diagnoses, we carried out a comparative study involving three distinct groups: (i) dermatologists, (ii) the AI model and (iii) dermatologists + AI model. The dermatologist group comprised 23 dermatologists with a certain level of expertise and more than 10 h of systematic dermoscopy training. We used RegNetY400MF as the backbone network to extract features from the dermoscopic and clinical images. Results The AI model demonstrated higher levels of total accuracy, precision, sensitivity and specificity in the classification of MF and other inflammatory skin diseases than participating dermatologists. A significant enhancement was noticed in the average accuracy, sensitivity and specificity for MF and inflammatory diseases in the ‘dermatologist + AI’ group, with values of 82.9%, 86.2% and 96.5%, respectively, compared with 71.5%, 74.6% and 94.1%, respectively, in the ‘dermatologist-only’ group. A more accurate diagnosis of each disease was also achieved by the multiclassification model. Conclusions The results indicate that our AI model has a significantly strong discriminative ability to assist dermatologists with improving diagnostic accuracy in early-stage MF and common inflammatory skin diseases.