BACT: nonparametric Bayesian cell typing for single-cell spatial transcriptomics data

Abstract The spatial transcriptomics is a rapidly evolving biological technology that simultaneously measures the gene expression profiles and the spatial locations of spots. With progressive advances, current spatial transcriptomic techniques can achieve the cellular or even the subcellular resolution, making it possible to explore the fine-grained spatial pattern of cell types within one tissue section. However, most existing cell spatial clustering methods require a correct specification of the cell type number, which is hard to determine in the practical exploratory data analysis. To address this issue, we present a nonparametric Bayesian model BACT to perform BAyesian Cell Typing by utilizing gene expression information and spatial coordinates of cells. BACT incorporates a nonparametric Potts prior to induce neighboring cells’ spatial dependency, and, more importantly, it can automatically learn the cell type number directly from the data without prespecification. Evaluations on three single-cell spatial transcriptomic datasets demonstrate the better performance of BACT than competing spatial cell typing methods. The R package and the user manual of BACT are publicly available at https://github.com/yinqiaoyan/BACT.