Machine learning algorithms performed no better than regression models for prognostication in traumatic brain injury
Benjamin Gravesteijn, Daan Nieboer, Ari Ercole, Hester F. Lingsma, David Nelson, Ben Van Calster, Ewout W. Steyerberg, Cecilia Åkerlund, Krisztina Amrein, Nada Anđelić, Lasse Andreassen, Audny Anke, Anna Antoni, Gérard Audibert, Philippe Azouvi, Maria Luisa Azzolini, Ronald Bartels, Pál Barzó, Romuald Beauvais, Ronny Beer, Bo‐Michael Bellander, Antonio Belli, Habib Benali, Maurizio Berardino, Luigi Beretta, Morten Blaabjerg, Peter Bragge, Alexandra Bražinová, Vibeke Brinck, Joanne Brooker, Camilla Brorsson, András Büki, Monika Bullinger, Manuel Cabeleira, Alessio Caccioppola, Emiliana Calappi, Maria Rosa Calvi, Peter Cameron, Guillermo Carbayo Lozano, Marco Carbonara, Giorgio Chevallard, Arturo Chieregato, Giuseppe Citerio, Maryse C. Cnossen, Mark Coburn, Jonathan Coles, D. James Cooper, Marta Correia, Amra Čović, Nicola Curry, Endre Czeiter, Marek Czosnyka, Claire Dahyot‐Fizelier, Helen Dawes, Véronique De Keyser, Vincent Degos, Françesco Della Corte, Hugo den Boogert, Bart Depreitere, Đula Đilvesi, Abhishek Dixit, Emma Donoghue, Jens Dreier Guy-Loup Dulière, Ari Ercole, Patrick Esser, Erzsébet Ezer Martin Fabricius, Kelly Foks Valery L. Feigin, Shirin Frisvold, Alex Furmanov, Pablo Gagliardo, Damien Galanaud, Dashiell Gantner, Guoyi Gao, Pradeep George, Alexandre Ghuysen, Lelde Giga, Ben Glocker, Jagoš Golubović, Pedro A. Gómez, Johannes Gratz, Benjamin Gravesteijn, Francesca Grossi, Russell L. Gruen, Deepak Gupta, Juanita A. Haagsma, Iain Haitsma, Raimund Helbok, Eirik Helseth, Lindsay Horton, Jilske Huijben, Peter J. Hutchinson, Bram Jacobs, Stefan Jankowski, Mike Jarrett Ji-yao Jiang, Kelly Jones, Mladen Karan, Angelos G. Kolias, Erwin J. O. Kompanje, Daniel Kondziella, Evgenios Koraropoulos
Abstract
OBJECTIVE: We aimed to explore the added value of common machine learning (ML) algorithms for prediction of outcome for moderate and severe traumatic brain injury. STUDY DESIGN AND SETTING: We performed logistic regression (LR), lasso regression, and ridge regression with key baseline predictors in the IMPACT-II database (15 studies, n = 11,022). ML algorithms included support vector machines, random forests, gradient boosting machines, and artificial neural networks and were trained using the same predictors. To assess generalizability of predictions, we performed internal, internal-external, and external validation on the recent CENTER-TBI study (patients with Glasgow Coma Scale <13, n = 1,554). Both calibration (calibration slope/intercept) and discrimination (area under the curve) was quantified. RESULTS: In the IMPACT-II database, 3,332/11,022 (30%) died and 5,233(48%) had unfavorable outcome (Glasgow Outcome Scale less than 4). In the CENTER-TBI study, 348/1,554(29%) died and 651(54%) had unfavorable outcome. Discrimination and calibration varied widely between the studies and less so between the studied algorithms. The mean area under the curve was 0.82 for mortality and 0.77 for unfavorable outcomes in the CENTER-TBI study. CONCLUSION: ML algorithms may not outperform traditional regression approaches in a low-dimensional setting for outcome prediction after moderate or severe traumatic brain injury. Similar to regression-based prediction models, ML algorithms should be rigorously validated to ensure applicability to new populations.