变压器
计算机科学
无监督学习
人工智能
语言模型
机器学习
蛋白质结构预测
工程类
蛋白质结构
生物
生物化学
电气工程
电压
作者
Roshan Rao,Joshua Meier,Tom Sercu,С. Г. Овчинников,Alexander Rives
标识
DOI:10.1101/2020.12.15.422761
摘要
A bstract Unsupervised contact prediction is central to uncovering physical, structural, and functional constraints for protein structure determination and design. For decades, the predominant approach has been to infer evolutionary constraints from a set of related sequences. In the past year, protein language models have emerged as a potential alternative, but performance has fallen short of state-of-the-art approaches in bioinformatics. In this paper we demonstrate that Transformer attention maps learn contacts from the unsupervised language modeling objective. We find the highest capacity models that have been trained to date already outperform a state-of-the-art unsupervised contact prediction pipeline, suggesting these pipelines can be replaced with a single forward pass of an end-to-end model. 1
科研通智能强力驱动
Strongly Powered by AbleSci AI