Validation of Deep Convolutional Neural Network-based algorithm for detection of diabetic retinopathy – Artificial intelligence versus clinician for screening

Purpose: Deep learning is a newer and advanced subfield in artificial intelligence (AI). The aim of our study is to validate a machine-based algorithm developed based on deep convolutional neural networks as a tool for screening to detect referable diabetic retinopathy (DR). Methods: An AI algorithm to detect DR was validated at our hospital using an internal dataset consisting of 1,533 macula-centered fundus images collected retrospectively and an external validation set using Methods to Evaluate Segmentation and Indexing Techniques in the field of Retinal Ophthalmology (MESSIDOR) dataset. Images were graded by two retina specialists as any DR, prompt referral (moderate nonproliferative diabetic retinopathy (NPDR) or above or presence of macular edema) and sight-threatening DR/STDR (severe NPDR or above) and compared with AI results. Sensitivity, specificity, and area under curve (AUC) for both internal and external validation sets for any DR detection, prompt referral, and STDR were calculated. Interobserver agreement using kappa value was calculated for both the sets and two out of three agreements for DR grading was considered as ground truth to compare with AI results. Results: In the internal validation set, the overall sensitivity and specificity was 99.7% and 98.5% for Any DR detection and 98.9% and 94.84%for Prompt referral respectively. The AUC was 0.991 and 0.969 for any DR detection and prompt referral respectively. The agreement between two observers was 99.5% and 99.2% for any DR detection and prompt referral with a kappa value of 0.94 and 0.96, respectively. In the external validation set (MESSIDOR 1), the overall sensitivity and specificity was 90.4% and 91.0% for any DR detection and 94.7% and 97.4% for prompt referral, respectively. The AUC was. 907 and. 960 for any DR detection and prompt referral, respectively. The agreement between two observers was 98.5% and 97.8% for any DR detection and prompt referral with a kappa value of 0.971 and 0.980, respectively. Conclusion: With increasing diabetic population and growing demand supply gap in trained resources, AI is the future for early identification of DR and reducing blindness. This can revolutionize telescreening in ophthalmology, especially where people do not have access to specialized health care.