Large-scale benchmarking of circRNA detection tools reveals large differences in sensitivity but not in precision

Abstract The detection of circular RNA molecules (circRNAs) is typically based on short-read RNA sequencing data processed by computational detection tools. During the last decade, a plethora of such tools have been developed, but a systematic comparison with orthogonal validation is missing. Here, we set up a circRNA detection tool benchmarking study, in which 16 tools were used and detected over 315,000 unique circRNAs in three deeply sequenced human cell types. Next, 1,516 predicted circRNAs were empirically validated using three orthogonal methods. Generally, tool-specific precision values are high and similar (median of 98.8%, 96.3%, and 95.5% for qPCR, RNase R, and amplicon sequencing, respectively) whereas the sensitivity and number of predicted circRNAs (ranging from 1,372 to 58,032) are the most significant tool differentiators. Furthermore, we demonstrate the complementarity of tools through the increase in detection sensitivity by considering the union of highly-precise tool combinations while keeping the number of false discoveries low. Finally, based on the benchmarking results, recommendations are put forward for circRNA detection and validation.