Artificial intelligence for antibody reading comprehension: AntiBERTa
Yoonjoo Choi
Abstract
Utilizing publicly available antibody big data resources, Leem et al. (2022) developed an antibody-specific language model, AntiBERTa, to understand the “language” of antibodies. Case studies reveal that AntiBERTa can be an extremely useful tool for antibody engineering. Utilizing publicly available antibody big data resources, Leem et al. (2022) developed an antibody-specific language model, AntiBERTa, to understand the “language” of antibodies. Case studies reveal that AntiBERTa can be an extremely useful tool for antibody engineering. There are a large number of players in the immune system to protect biological individuals from harmful foreign substances. Among those, the B cell is a main player in the adaptive immune system. B cells are equipped with receptor molecules (B cell receptor) that can be secreted upon activation. The secreted molecules, antibody, are known to be astronomically diverse (estimated 1013–1015). The high diversity of the antibody is a two-faced Janus. The immune system can respond to nearly any type of antigen, mainly due to the large diversity of antibodies. According to Antibodypedia,1Björling E. Uhlén M. Antibodypedia, a portal for sharing antibody and antigen validation data.Mol. Cell. Proteomics. 2008; 7: 2028-2037https://doi.org/10.1074/mcp.m800264-mcp200Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar 4.6 million monoclonal antibodies are currently available for 19,000 genes. The diversity also enables antibodies to be highly successful as biotherapeutics. In 2021, FDA approved the 100th therapeutic antibody.2Mullard A. FDA approves 100th monoclonal antibody product.Nat. Rev. Drug Discov. 2021; 20: 491-495https://doi.org/10.1038/d41573-021-00079-7Crossref PubMed Scopus (85) Google Scholar The coronavirus pandemic has been currently boosting the development of therapeutic antibodies for COVID-19, and several new antibodies are waiting to treat SARS-CoV-2-infected patients. On the other hand, such rich diversity may not be always advantageous. Despite the fact that antibodies have been (perhaps the most) extensively studied and the antibody-related biopharmaceutical industry continues to mature, there seem to be a lot of things to learn about antibodies, as evidenced in the increasing growth of papers with the publication keyword, “antibody.” It is simply practically impossible to experimentally explore the entire antibody repertoire. Thus, computational approaches using artificial intelligence (AI) techniques have become essential for antibody research. The advancement of AI and big data are not separable. Recent advances in next-generation sequencing technology now enable the construction of a large volume of antibody repertoires. The observed antibody space (OAS) database3Kovaltsuk A. Leem J. Kelm S. Snowden J. Deane C.M. Krawczyk K. Observed antibody space: a resource for data mining next-generation sequencing of antibody repertoires.J. Immunol. 2018; 201: 2502-2509https://doi.org/10.4049/jimmunol.1800708Crossref PubMed Scopus (59) Google Scholar,4Olsen T.H. Boyles F. Deane C.M. Observed Antibody Space: a diverse database of cleaned, annotated, and translated unpaired and paired antibody sequences.Protein Sci. 2022; 31: 141-146https://doi.org/10.1002/pro.4205Crossref Scopus (4) Google Scholar is a compilation of known repertoire studies and databases. Since the release of OAS, many practical applications have been developed including computational antibody humanization using AI.5Marks C. Hummer A.M. Chin M. Deane C.M. Humanization of antibodies using a machine learning approach on large-scale repertoire data.Bioinformatics. 2021; 37: 4041-4047https://doi.org/10.1093/bioinformatics/btab434Crossref Google Scholar,6Prihoda D. Maamary J. Waight A. Juan V. Fayadat-Dilman L. Svozil D. Bitton D.A. BioPhi: a platform for antibody design, humanization, and humanness evaluation based on natural antibody repertoires and deep learning.mAbs. 2022; 14: 2020203https://doi.org/10.1080/19420862.2021.2020203Crossref Scopus (4) Google Scholar The antibody repertoire big data resources also provide an in-depth view and biological insights into antibodies.7Marks C. Deane C.M. How repertoire data are changing antibody science.J. Biol. Chem. 2020; 295: 9823-9837https://doi.org/10.1074/jbc.rev120.010181Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar Here, Leem et al. present an antibody-specific language model in a timely manner. AntiBERTa (antibody-specific bidirectional encoder representation from transformers) is a 12-layer transformer model pre-trained using the OAS database.8Leem J. Mitchell L.S. Farmery J.H. Barton J. Galson J.D. Deciphering the Language of Antibodies Using Self-Supervised Learning.Patterns. 2022; 3: 100513Abstract Full Text Full Text PDF Scopus (1) Google Scholar In fact, there has been a language model for general proteins9Elnaggar A. Heinzinger M. Dallago C. Rehawi G. Wang Y. Jones L. Gibbs T. Feher T. Angerer C. Steinegger M. et al.ProtTrans: towards cracking the language of Life's code through self-supervised deep learning and high performance computing.IEEE Trans. Pattern Anal. Mach. Intell. 2021; : 1https://doi.org/10.1109/TPAMI.2021.3095381Crossref Scopus (40) Google Scholar (ProtBERT), and there have been other antibody-specific language models, such as DeepAb10Ruffolo J.A. Sulam J. Gray J.J. Antibody structure prediction using interpretable deep learning.Patterns. 2022; 3: 100406https://doi.org/10.1016/j.patter.2021.100406Abstract Full Text Full Text PDF PubMed Scopus (10) Google Scholar and Sapiens.6Prihoda D. Maamary J. Waight A. Juan V. Fayadat-Dilman L. Svozil D. Bitton D.A. BioPhi: a platform for antibody design, humanization, and humanness evaluation based on natural antibody repertoires and deep learning.mAbs. 2022; 14: 2020203https://doi.org/10.1080/19420862.2021.2020203Crossref Scopus (4) Google Scholar Comparing with those existing methods, however, AntiBERTa is more versatile and specific with deeper layers. It is remarkable that AntiBERTa nicely partitions memory and naive B cells, whereas other models showed relatively less distinct results; i.e. the antibody-specific deep-layered model indeed learns the language of antibodies and finds the origin of B cell. One of the direct applications can be the estimation of antibody humanness and immunogenicity for the development of safer therapeutic antibodies. It is well known that antibodies with high human content tend to be less immunogenic. As demonstrated in the separation of memory and naive B cells, AntiBERTa is shown to be successful in classifying their species origin (murine, chimeric, humanized, and human). The antibody-specific model generally provides better descriptions of antibodies than the general protein model. The authors found that residue pairs with high self-attention scores give structural insights into long-range interactions, which were not identified by the general protein model. The insight naturally leads to the prediction of paratopes, antigen binding sites. From several case studies, the authors showed that AntiBERTa successfully identifies paratope residues that are not in complementarity determining regions (CDR). While the authors demonstrated that AntiBERTa outperforms other methods and provided convincing rationales, they also leave something to be desired. As the authors stated in the main manuscript, AntiBERTa can be directly applicable to antibody-structure prediction and humanization (or both at the same time), but the authors left it as potential applications. In the near future, we hope to meet practical application tools based on the AntiBERTa model. This work was supported by the National Research Foundation of Korea (NRF) grants funded by the Korean government ( MSIT ) ( NRF-2020R1A5A2031185 and NRF-2020M3A9G3080281 ). The author declares no competing interests. Deciphering the language of antibodies using self-supervised learningLeem et al.PatternsMay 17, 2022In BriefAntibodies are guardians of the adaptive immune system, with over one billion variants in one individual. Understanding antibody function is critical for deciphering the biology of disease and for discovering novel therapeutics. Here, we present AntiBERTa, a deep-language model that learns the features and syntax, or “language,” of antibodies. We demonstrate the model’s capacity through a range of tasks, such as tracing the B cell origin of the antibody, quantifying immunogenicity, and predicting the antibody’s binding site. Full-Text PDF Open Access