Benchmarking solutions to the T-cell receptor epitope prediction problem: IMMREP22 workshop report
Pieter Meysman, Justin Barton, Barbara Bravi, Liel Cohen-Lavi, В. К. Карнаухов, Elias Lilleskov, Alessandro Montemurro, Morten Nielsen, Thierry Mora, Paul Pereira, Anna Postovskaya, María Rodríguez Martínez, Jorge Fernández-de-Cossio-Díaz, Alexandra Vujkovic, Aleksandra M. Walczak, Anna Weber, Rose Yin, Anne Eugster, Virag Sharma
Abstract
Many different solutions to predicting the cognate epitope target of a T-cell receptor (TCR) have been proposed. However several questions on the advantages and disadvantages of these different approaches remain unresolved, as most methods have only been evaluated within the context of their initial publications and data sets. Here, we report the findings of the first public TCR-epitope prediction benchmark performed on 23 prediction models in the context of the ImmRep 2022 TCR-epitope specificity workshop. This benchmark revealed that the use of paired-chain alpha-beta, as well as CDR1/2 or V/J information, when available, improves classification obtained with CDR3 data, independent of the underlying approach. In addition, we found that straight-forward distance-based approaches can achieve a respectable performance when compared to more complex machine-learning models. Finally, we highlight the need for a truly independent follow-up benchmark and provide recommendations for the design of such a next benchmark.