Implementation and Updating of Clinical Prediction Models: A Systematic Review
Alexander Saelmans, Tom M Seinen, Victor Pera, Aniek F. Markus, Egill A. Friðgeirsson, Luis H. John, Lieke Schiphof-Godart, Peter R. Rijnbeek, Jenna Reps, Ross D. Williams
Abstract
Objective: To summarize the implementation approaches and updating methods of clinically implemented models and consecutively advise researchers on the implementation and updating. Patients and Methods: We included studies describing the implementation of prognostic binary prediction models in a clinical setting. We retrieved articles from Embase, Medline, and Web of Science from January 1, 2010, to January 1, 2024. We performed data extraction, based on Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis and Prediction Model Risk of Bias Assessment guidelines, and summarized. Results: The search yielded 1872 articles. Following screening, 37 articles, describing 56 prediction models, were eligible for inclusion. The overall risk of bias was high in 86% of publications. In model development and internal validation, 32% of the models was assessed for calibration. External validation was performed for 27% of the models. Most models were implemented into the hospital information system (63%), followed by a web application (32%) and a patient decision aid tool (5%). Moreover, 13% of models have been updated following implementation. Conclusion: Impact assessments generally showed successful model implementation and the ability to improve patient care, despite not fully adhering to prediction modeling best practice. Both impact assessment and updating could play a key role in identifying and lowering bias in models.