Random forest machine learning method outperforms prehospital National Early Warning Score for predicting one-day mortality: A retrospective study
Jussi Pirneskoski, Joonas Tamminen, Antti Kallonen, Jouni Nurmi, Markku Kuisma, Klaus T. Olkkola, Sanna Hoppu
Abstract
AIM OF THE STUDY: The National Early Warning Score (NEWS) is a validated method for predicting clinical deterioration in hospital wards, but its performance in prehospital settings remains controversial. Modern machine learning models may outperform traditional statistical analyses for predicting short-term mortality. Thus, we aimed to compare the mortality prediction accuracy of NEWS and random forest machine learning using prehospital vital signs. METHODS: In this retrospective study, all electronic ambulance mission reports between 2008 and 2015 in a single EMS system were collected. Adult patients (≥ 18 years) were included in the analysis. Random forest models with and without blood glucose were compared to the traditional NEWS for predicting one-day mortality. A ten-fold cross-validation method was applied to train and validate the random forest models. RESULTS: A total of 26,458 patients were included in the study of whom 278 (1.0%) died within one day of ambulance mission. The area under the receiver operating characteristic curve for one-day mortality was 0.836 (95% CI, 0.810-0.860) for NEWS, 0.858 (95% CI, 0.832-0.883) for a random forest trained with NEWS variables only and 0.868 (0.843-0.892) for a random forest trained with NEWS variables and blood glucose. CONCLUSION: A random forest algorithm trained with NEWS variables was superior to traditional NEWS for predicting one-day mortality in adult prehospital patients, although the risk of selection bias must be acknowledged. The inclusion of blood glucose in the model further improved its predictive performance.