CAGI, the Critical Assessment of Genome Interpretation, establishes progress and prospects for computational genetic variant interpretation methods
Shantanu Jain, Constantina Bakolitsa, Steven E. Brenner, Predrag Radivojac, John Moult, Susanna Repo, Roger A. Hoskins, Gaia Andreoletti, Daniel Barsky, Ajithavalli Chellapan, Hoyin Chu, Navya Dabbiru, Naveen K. Kollipara, Melissa Ly, Andrew J. Neumann, Lipika R. Pal, Eric Odell, Gaurav Pandey, Robin C. Peters-Petrulewicz, Rajgopal Srinivasan, Stephen F. Yee, Sri Jyothsna Yeleswarapu, Maya Zuhl, Ogün Adebalı, Ayoti Patra, M Beer, Raghavendra Hosur, Jian Peng, Brady Bernard, M. Victoria Berry, Shengcheng Dong, Alan P. Boyle, Aashish N. Adhikari, Jingqi Chen, Zhiqiang Hu, Robert Wang, Yaqiong Wang, Max J. Miller, Yanran Wang, Yana Bromberg, Paola Turina, Emidio Capriotti, James J. Han, Kıvılcım Öztürk, Hannah Carter, Giulia Babbi, Samuele Bovo, Pietro Di Lena, Pier Luigi Martelli, Castrense Savojardo, Rita Casadio, Melissa Cline, Greet De Baets, Sandra Bonache, Orland Dı́ez, Sara Gutiérrez‐Enríquez, Alejandro Fernández, Gemma Montalban, Lars Ootes, Selen Özkan, Natàlia Padilla, Casandra Riera, Xavier de la Cruz, Mark Diekhans, Peter J. Huwe, Qiong Wei, Qifang Xu, Roland L. Dunbrack, Valer Gotea, Laura Elnitski, Gennady Margolin, Piero Fariselli, Ivan V. Kulakovskiy, Vsevolod J. Makeev, Dmitry Penzar, Ilya E. Vorontsov, Alexander V. Favorov, Julia Forman, Marcia A. Hasenahuer, Marı́a Silvina Fornasari, Gustavo Parisi, Žiga Avsec, Muhammed Hasan Çelik, Thi Yen Duong Nguyen, Julien Gagneur, Fangyuan Shi, Matthew D. Edwards, Yuchun Guo, Kevin Tian, Haoyang Zeng, David K. Gifford, Jonathan Göke, Jan Zaucha, Julian Gough, Graham R. S. Ritchie, Adam Frankish, Jonathan M. Mudge, Jennifer Harrow, Erin L. Young, Yao Yu
Abstract
BACKGROUND: The Critical Assessment of Genome Interpretation (CAGI) aims to advance the state-of-the-art for computational prediction of genetic variant impact, particularly where relevant to disease. The five complete editions of the CAGI community experiment comprised 50 challenges, in which participants made blind predictions of phenotypes from genetic data, and these were evaluated by independent assessors. RESULTS: Performance was particularly strong for clinical pathogenic variants, including some difficult-to-diagnose cases, and extends to interpretation of cancer-related variants. Missense variant interpretation methods were able to estimate biochemical effects with increasing accuracy. Assessment of methods for regulatory variants and complex trait disease risk was less definitive and indicates performance potentially suitable for auxiliary use in the clinic. CONCLUSIONS: Results show that while current methods are imperfect, they have major utility for research and clinical applications. Emerging methods and increasingly large, robust datasets for training and assessment promise further progress ahead.