Predicting Endometrial Cancer Subtypes and Molecular Features from Histopathology Images Using Multi-resolution Deep Learning Models

Determining endometrial carcinoma histological subtype is a critical diagnostic process that directly affects prognosis and treatment options. Recently, molecular subtyping and mutation status are gaining popularity as they offer more relevant information to evaluate the severity and develop individualized therapies. However, compared to the histopathological approach, the availability of molecular subtyping is limited as it can only be obtained by genomic sequencing, which is more expensive. Here, we implemented deep convolutional neural network models that predict not only the histological subtypes, but also molecular subtypes and 18 common gene mutations based on digitized H&E stained pathological images. Taking advantage of the multi-resolution nature of the whole slide images, we introduced a customized architecture, Panoptes, to integrate features of different magnification. The model was trained and evaluated with images from The Cancer Genome Atlas and Clinical Proteomic Tumor Analysis Consortium. Our models achieved an area under the receiver operating characteristic curve (AUROC) of 0.969 in predicting histological subtype and 0.934 to 0.958 in predicting CNV-H molecular subtype. The prediction tasks of 4 mutations and MSI-High molecular subtype also achieved high performance with AUROC ranging from 0.781 to 0.873. Panoptes showed significantly better performance than InceptionResnet in most of these top predicted tasks by up to 18%. Feature extraction and visualization revealed that the model relied on human-interpretable patterns. Our results suggest that Panoptes can help pathologists determine molecular subtypes and mutations without sequencing, and is generally applicable to any cancer type.