Structure Aware Large Language Models for Accurate Prediction of RNA Secondary Structures

This study presents a structure-aware large language model for accurate prediction of RNA secondary structures directly from primary sequence. The model extends a transformer backbone with a dual-stream architecture that jointly encodes nucleotides and dot-bracket tokens, a grammar-constrained decoder that enforces context-free validity, and a graph-informed attention module that injects candidate base-pair priors during inference. Training is performed on 1.36×10⁶ sequence-structure pairs spanning 203±11 RNA families with lengths from 40 to 1,024 nucleotides. Evaluation on non-overlapping family-level splits shows that the proposed model achieves base-pair F1 scores of 0.884±0.012 on medium-length and 0.837±0.017 on long RNAs, outperforming thermodynamic and neural baselines by 0.041-0.079 absolute F1 while keeping bracket validity above 0.998±0.001. Ablation experiments indicate that grammar constraints account for 63.2±7.4% of the gain in structural validity, whereas graph fusion explains 58.9±6.1% of the improvement on long-range motifs. Despite richer structural bias, decoding throughput reaches 1.27×10³±84 sequences per second on commodity GPUs.