GenerRNA: A generative pre-trained language model for de novo RNA design

A bstract The design of RNA plays a crucial role in developing RNA vaccines, nucleic acid therapeutics, and innovative biotechnological tools. Nevertheless, existing techniques lack versatility across various tasks and frequently suffer from a deficiency of automated generation. Inspired by the remarkable success of Large Language Models (LLMs) in the realm of protein and molecule design, we present GenerRNA, the first large-scale pre-trained model for RNA generation, aiming to further automate RNA design. Our approach eliminates the need for secondary structure or other prior knowledge and is capable of de novo generation of RNA with stable secondary structures while ensuring its distinctiveness from existing sequences. This widens our exploration of RNA space, thereby enriching our understanding of RNA structures and functions. Moreover, GenerRNA is fine-tunable on smaller, more specialized datasets for particular subtasks. This flexibility and versatility enables the generation of RNAs with desired specific functionalities or properties. Upon fine-tuning GenerRNA, we successfully generated novel RNA sequences exhibiting high affinity for target proteins. GenerRNA is freely available at the following repository: https://github.com/pfnet-research/GenerRNA