Foundation models of protein sequences: A brief overview
Andreas Bjerregaard, Peter Mørch Groth, Søren Hauberg, Anders Krogh, Wouter Boomsma
Abstract
Protein sequence models have evolved from simple statistics of aligned families to versatile foundation models of evolutionary scale. Enabled by self-supervised learning and an abundance of protein sequence data, such foundation models now play a central role in protein science. They facilitate rich representations, powerful generative design, and fine-tuning across diverse domains. In this review, we trace modeling developments and categorize them into methodological trends over the modalities they describe and the contexts they condition upon. Following a brief historical overview, we focus our attention on the most recent trends and outline future perspectives. • A review of three decades of statistical protein modeling. • The key difference between models lies in how they model amino acid context . • Protein foundation models are becoming increasingly multi-modal. • Fine-tuned models outperform fixed protein representations. • Further scaling shows tendencies of diminishing returns.