Crowd-sourced benchmarking of single-sample tumor subclonal reconstruction
Adriana Salcedo, Maxime Tarabichi, Alex Buchanan, Shadrielle M. G. Espiritu, Hongjiu Zhang, Kaiyi Zhu, Tai-Hsien Ou Yang, Ignaty Leshchiner, Dimitris Anastassiou, Yuanfang Guan, Gun Ho Jang, Mohammed Faizal Eeman Mootor, Kerstin Haase, Amit G. Deshwar, William Y. Zou, Imaad Umar, Stefan C. Dentro, Jeff Wintersinger, Kami Chiotti, Jonas Demeulemeester, Clemency Jolly, Lesia Sycza, Minjeong Ko, PCAWG Evolution and Heterogeneity Working Group, Stefan C. Dentro, Ignaty Leshchiner, Moritz Gerstung, Maxime Tarabichi, Jeff Wintersinger, Amit G. Deshwar, Kaixian Yu, Santiago González, Yulia Rubanova, Geoff Macintyre, David J. Adams, Pavana Anur, Rameen Beroukhim, Paul C. Boutros, David D.L. Bowtell, Peter J. Campbell, Shaolong Cao, Elizabeth L. Christie, Marek Cmero, Yupeng Cun, Kevin J. Dawson, Nilgun Donmez, Ruben M. Drews, Roland Eils, Yu Fan, Matthew W. Fittall, Dale W. Garsed, Gad Getz, Gavin Ha, Marcin Imieliński, Lara Jerman, Yuan Ji, Kortine Kleinheinz, Juhee Lee, Henry Lee-Six, Dimitri Livitz, Salem Malikic, Florian Markowetz, Iñigo Martincorena, Thomas J. Mitchell, Ville Mustonen, Layla Oesper, Martin Peifer, Myron Peto, Benjamin J. Raphael, Daniel Rosebrock, S. Cenk Şahinalp, Adriana Salcedo, Matthias Schlesner, Steven E. Schumacher, Subhajit Sengupta, Ruian Shi, Seung Jun Shin, Lincoln D. Stein, Oliver Spiro, Ignacio Vázquez-Garćıa, Shankar Vembu, David A. Wheeler, Tsun-Po Yang, Xiaotong Yao, Ke Yuan, Hongtu Zhu, Wenyi Wang, Quaid Morris, Paul T. Spellman, David C. Wedge, Peter Van Loo, SMC-Het Participants, Alokkumar Jha, Tanxiao Huang, Tsun-Po Yang, Martin Peifer, S. Cenk Şahinalp, Salem Malikic, Ignacio Vázquez-Garćıa, Ville Mustonen
Abstract
Subclonal reconstruction algorithms use bulk DNA sequencing data to quantify parameters of tumor evolution, allowing an assessment of how cancers initiate, progress and respond to selective pressures. We launched the ICGC-TCGA (International Cancer Genome Consortium-The Cancer Genome Atlas) DREAM Somatic Mutation Calling Tumor Heterogeneity and Evolution Challenge to benchmark existing subclonal reconstruction algorithms. This 7-year community effort used cloud computing to benchmark 31 subclonal reconstruction algorithms on 51 simulated tumors. Algorithms were scored on seven independent tasks, leading to 12,061 total runs. Algorithm choice influenced performance substantially more than tumor features but purity-adjusted read depth, copy-number state and read mappability were associated with the performance of most algorithms on most tasks. No single algorithm was a top performer for all seven tasks and existing ensemble strategies were unable to outperform the best individual methods, highlighting a key research need. All containerized methods, evaluation code and datasets are available to support further assessment of the determinants of subclonal reconstruction accuracy and development of improved methods to understand tumor evolution.