A scalable reinforcement learning approach for screening large peptide libraries for bioactive peptide discovery

Abstract Bioactive peptides such as anticancer peptides (ACPs) offer a promising therapeutic alternative to small molecules due to their efficiency and selectivity against tumors and minimal toxicity towards healthy human cells. However, their rational discovery requires navigating a vast chemical space using computationally demanding in silico tools. Herein, we present a computational method enabling cost-efficient exploration of large peptide libraries using reinforcement learning and posterior sampling. Practical application of the developed approach results in identification of membranolytic peptides with therapeutic potential. The developed computational method reduces the search space by over 90% compared to exhaustive library screening and enables effective balancing between dataset’s exploration and exploitation. We demonstrate the scalability of this method by screening a focused library of 36 million structurally resolved helical peptides curated from the Protein Data Bank. When screened in in vitro assays, 15 of the top 100 selected candidates exhibit cytotoxic activity against breast cancer cells including drug resistant triple-negative breast cancer, with the three lead compounds further characterizing as non-toxic towards healthy human cells. This study highlights the potential of using deep reinforcement learning to expedite bioactive peptide discovery, offering a promising path for developing new peptide-based cancer therapies.