Screening and identifying hepatobiliary diseases through deep learning using ocular images: a prospective, multicentre study

BACKGROUND: Ocular changes are traditionally associated with only a few hepatobiliary diseases. These changes are non-specific and have a low detection rate, limiting their potential use as clinically independent diagnostic features. Therefore, we aimed to engineer deep learning models to establish associations between ocular features and major hepatobiliary diseases and to advance automated screening and identification of hepatobiliary diseases from ocular images. METHODS: We did a multicentre, prospective study to develop models using slit-lamp or retinal fundus images from participants in three hepatobiliary departments and two medical examination centres. Included participants were older than 18 years and had complete clinical information; participants diagnosed with acute hepatobiliary diseases were excluded. We trained seven slit-lamp models and seven fundus models (with or without hepatobiliary disease [screening model] or one specific disease type within six categories [identifying model]) using a development dataset, and we tested the models with an external test dataset. Additionally, we did a visual explanation and occlusion test. Model performances were evaluated using the area under the receiver operating characteristic curve (AUROC), sensitivity, specificity, and F1* score. FINDINGS: Between Dec 16, 2018, and July 31, 2019, we collected data from 1252 participants (from the Department of Hepatobiliary Surgery of the Third Affiliated Hospital of Sun Yat-sen University, the Department of Infectious Diseases of the Affiliated Huadu Hospital of Southern Medical University, and the Nantian Medical Centre of Aikang Health Care [Guangzhou, China]) for the development dataset; between Aug 14, 2019, and Jan 31, 2020, we collected data from 537 participants (from the Department of Infectious Diseases of the Third Affiliated Hospital of Sun Yat-sen University and the Huanshidong Medical Centre of Aikang Health Care [Guangzhou, China]) for the test dataset. The AUROC for screening for hepatobiliary diseases of the slit-lamp model was 0·74 (95% CI 0·71-0·76), whereas that of the fundus model was 0·68 (0·65-0·71). For the identification of hepatobiliary diseases, the AUROCs were 0·93 (0·91-0·94; slit-lamp) and 0·84 (0·81-0·86; fundus) for liver cancer, 0·90 (0·88-0·91; slit-lamp) and 0·83 (0·81-0·86; fundus) for liver cirrhosis, and ranged 0·58-0·69 (0·55-0·71; slit-lamp) and 0·62-0·70 (0·58-0·73; fundus) for other hepatobiliary diseases, including chronic viral hepatitis, non-alcoholic fatty liver disease, cholelithiasis, and hepatic cyst. In addition to the conjunctiva and sclera, our deep learning model revealed that the structures of the iris and fundus also contributed to the classification. INTERPRETATION: Our study established qualitative associations between ocular features and major hepatobiliary diseases, providing a non-invasive, convenient, and complementary method for hepatobiliary disease screening and identification, which could be applied as an opportunistic screening tool. FUNDING: Science and Technology Planning Projects of Guangdong Province; National Key R&D Program of China; Guangzhou Key Laboratory Project; National Natural Science Foundation of China.