Evaluating the impact of artificial intelligence in antimicrobial stewardship: a comparative meta-analysis with traditional risk scoring systems
Antonio Pinto, Flavia Pennisi, Giovanni Emanuele Ricciardi, Carlo Signorelli, Vincenza Gianfredi
Abstract
OBJECTIVES: The growing challenge of antimicrobial resistance (AMR) has underscored the urgent need for robust antimicrobial stewardship programs (AMS). Artificial intelligence (AI) and machine learning (ML) have emerged as promising tools to support enhanced decision-making in AMS. This systematic review and meta-analysis aims to evaluate the impact of AI in AMS and compare its effectiveness with traditional risk systems. METHODS: PubMed/MEDLINE, Scopus, EMBASE, and Web of Science were searched to identify studies published up to July 2024. Any studies that evaluated the use of AI/ML in AMS compared with conventional decision-making approaches were eligible. Outcomes of interested were predictive performance metrics and diagnostic accuracy. The meta-estimate was performed pooling standardized mean difference, and effect size (ES) measured as Cohen's d with a 95% confidence interval (CI). The risk of bias was assessed using the QUADAS-AI tool. RESULTS: Out of 3,458 studies, 27 were included, demonstrating that ML models outperform traditional methods in terms of sensitivity [1.93 (0.48-3.39) p = 0.009], and negative predictive value [1.66 (0.86-2.46), p < 0.001] but not in terms of area under the curve, accuracy, specificity, positive predictive value, when random effect models were applied. CONCLUSIONS: Our results revealed that ML tools offer promising enhancements to traditional AMS strategies. However, high heterogeneity, inconsistent results between fixed and random effect models, and limited use of external validation in retrieved studies raise concerns about the generalizability of the findings. Furthermore, the lack of representation from outpatient and pediatric settings highlights a critical equity gap in the application of these technologies.