External validation of the RISC, RISC-Malawi, and PERCH clinical prediction rules to identify risk of death in children hospitalized with pneumonia
Chris A. Rees, Shubhada Hooli, Carina King, Eric D. McCollum, Tim Colbourn, Norman Lufesi, Charles Mwansambo, Marzia Lazzerini, Shabir A. Madhi, Clare Cutland, Marta C. Nunes, Bradford D. Gessner, Sudha Basnet, Cissy B. Kartasasmita, Joseph L. Mathew, Syed Mohammad Akram uz Zaman, Gláucia Paranhos‐Baccalà, Shinjini Bhatnagar, Nitya Wadhwa, Rakesh Lodha, Satinder Aneja, Mathuram Santosham, Valentina Picot, Mariam Sylla, Shally Awasthi, Ashish Bavdekar, Jean W. Pape, Vanessa Rouzier, Monidarin Chou, Mala Rakoto‐Andrianarivelo, Jianwei Wang, Pagbajabyn Nymadawa, Philippe Vanhems, Graciela Russomando, Rai Muhammad Asghar, Salem Banajeh, Imran Iqbal, William MacLeod, Irene Maulén-Radován, G Miño, Samir K. Saha, Sunit Singhi, Donald M. Thea, Alexey Clara, Harry Campbell, Harish Nair, Jennifer Falconer, Linda Williams, Margaret Horne, Tor A. Strand, Shamim Qazi, Yasir Bin Nisar, Mark I. Neuman
Abstract
BACKGROUND: Existing scores to identify children at risk of hospitalized pneumonia-related mortality lack broad external validation. Our objective was to externally validate three such risk scores. METHODS: We applied the Respiratory Index of Severity in Children (RISC) for HIV-negative children, the RISC-Malawi, and the Pneumonia Etiology Research for Child Health (PERCH) scores to hospitalized children in the Pneumonia REsearch Partnerships to Assess WHO REcommendations (PREPARE) data set. The PREPARE data set includes pooled data from 41 studies on pediatric pneumonia from across the world. We calculated test characteristics and the area under the curve (AUC) for each of these clinical prediction rules. RESULTS: The RISC score for HIV-negative children was applied to 3574 children 0-24 months and demonstrated poor discriminatory ability (AUC = 0.66, 95% confidence interval (CI) = 0.58-0.73) in the identification of children at risk of hospitalized pneumonia-related mortality. The RISC-Malawi score had fair discriminatory value (AUC = 0.75, 95% CI = 0.74-0.77) among 17 864 children 2-59 months. The PERCH score was applied to 732 children 1-59 months and also demonstrated poor discriminatory value (AUC = 0.55, 95% CI = 0.37-0.73). CONCLUSIONS: In a large external application of the RISC, RISC-Malawi, and PERCH scores, a substantial number of children were misclassified for their risk of hospitalized pneumonia-related mortality. Although pneumonia risk scores have performed well among the cohorts in which they were derived, their performance diminished when externally applied. A generalizable risk assessment tool with higher sensitivity and specificity to identify children at risk of hospitalized pneumonia-related mortality may be needed. Such a generalizable risk assessment tool would need context-specific validation prior to implementation in that setting.