Is the ACS‐NSQIP Risk Calculator Accurate in Predicting Adverse Postoperative Outcomes in the Emergency Setting? An Italian Single‐center Preliminary Study
Giovanni Scotton, Giulio Del Zotto, L Bernardi, Annalisa Zucca, Susanna Terranova, Stefano Fracon, Lucia Paiano, Davide Cosola, Alan Biloslavo, Nicolò de Manzini
Abstract
BACKGROUND: The ACS-NSQIP surgical risk calculator (SRC) is an open-access online tool that estimates the chance for adverse postoperative outcomes. The risk is estimated based on 21 patient-related variables and customized for specific surgical procedures. The purpose of this monocentric retrospective study is to validate its predictive value in an Italian emergency setting. METHODS: From January to December 2018, 317 patients underwent surgical procedures for acute cholecystitis (n = 103), appendicitis (n = 83), gastrointestinal perforation (n = 45), and intestinal obstruction (n = 86). Patients' personal risk was obtained and divided by the average risk to calculate a personal risk ratio (RR). Areas under the ROC curves (AUC) and Brier score were measured to assess both the discrimination and calibration of the predictive model. RESULTS: The AUC was 0.772 (95%CI 0.722-0.817, p < 0.0001; Brier 0.161) for serious complications, 0.887 (95%CI 0.847-0.919, p < 0.0001; Brier 0.072) for death, and 0.887 (95%CI 0.847-0.919, p < 0.0001; Brier 0.106) for discharge to nursing or rehab facility. Pneumonia, cardiac complications, and surgical site infection presented an AUC of 0.794 (95%CI 0.746-0.838, p < 0.001; Brier 0.103), 0.836 (95%CI 0.790-0.875, p < 0.0001; Brier 0.081), and 0.729 (95%CI 0.676-0.777, p < 0.0001; Brier 0.131), respectively. A RR > 1.24, RR > 1.52, and RR > 2.63 predicted the onset of serious complications (sensitivity = 60.47%, specificity = 64.07%; NPV = 81%), death (sensitivity = 82.76%, specificity = 62.85%; NPV = 97%), and discharge to nursing or rehab facility (sensitivity = 80.00%, specificity = 69.12%; NPV = 95%), respectively. CONCLUSIONS: The calculator appears to be accurate in predicting adverse postoperative outcomes in our emergency setting. A RR cutoff provides a much more practical method to forecast the onset of a specific type of complication in a single patient.