Insitutype: likelihood-based cell typing for single cell spatial transcriptomics

Abstract Accurate cell typing is fundamental to analysis of spatial single-cell transcriptomics, but legacy scRNA-seq algorithms can underperform in this new type of data. We have developed a cell typing algorithm, Insitutype, designed for statistical and computational efficiency in spatial transcriptomics data. Insitutype is based on a likelihood model that weighs the evidence from every expression value, extracting all the information available in each cell’s expression profile. This likelihood model underlies a Bayes classifier for supervised cell typing, and an Expectation-Maximization algorithm for unsupervised and semi-supervised clustering. Insitutype also leverages alternative data types collected in spatial studies, such as cell images and spatial context, by using them to inform prior probabilities of cell type calls. We demonstrate rapid clustering of millions of cells and accurate fine-grained cell typing of kidney and non-small cell lung cancer samples.