Writing for Pediatric Critical Care Medicine: A Checklist When Using Administrative and Clinical Databases for Research

Abstract

Pediatric Critical Care Medicine (PCCM) contains a great deal of research that uses large, multicenter, curated datasets. Three resources stand out because they are frequently accessed by PCCM researchers. The Pediatric Health Information System (PHIS) dataset of a children’s hospitals network in the United States, which contains administrative information about inpatient encounters, including the PICU (https://www.childrenshospitals.org/content/analytics/product-program/pediatric-health-information-system). The Virtual Pediatric Systems, LLC (VPS) registry of collaborating U.S. PICUs, which has patient data about diagnosis, physiology, and outcome (https://myvps.org). The Extracorporeal Life Support Organization (ELSO) international dataset of patients of any age supported with extracorporeal membrane oxygenation (ECMO), with a focus on patient physiology, device type, and outcomes information (https://elso.org/aboutus.aspx). These, and other curated datasets, provide the epidemiology of our field and this research has potential for being informative, hypothesis-generating, and much valued–see the summary of PCCM’s 25-year highlights (1). However, there are precautions to consider when carrying out such work, particularly if there will be multiple reports from the same dataset. So, this sixth article in the “Writing for PCCM” series (2–6) is written to help researchers, authors, and readers best navigate this content in the Journal. WHY WE NEED A CHECKLIST FOR DATABASE COHORT STUDIES Cohort studies are nonexperimental in that patients have been selected based on a prior exposure, which could include admission to a children’s hospital or PICU, or use of life support like ECMO. As with any retrospective investigation, even if the dataset is prospectively curated, by the time a research question has been formulated all that can be known about an exposure and outcome has already been collected. There may be missing data. There is also risk that more than one research output with similar aims may come from the dataset. At present, not all journals have guidelines for authors reporting database research, and up to now PCCM has been in this category. However, PCCM needs to change its strategy for three reasons. First, the number of submissions is increasing. Consider publications since the beginning of 2022, PCCM has a collection of over 30 articles just from PHIS, VPS, and ELSO. Second, the Journal needs to help reviewers by guiding researchers. Third, the Journal must provide the best material for readers. Therefore, this new Writing for PCCM article presents a Checklist for authors that is loosely based on a JAMA Surgery (Journal of the American Medical Association Surgery) editorial in 2018 (7), with the addition of examples from PCCM publications. THE CHECKLIST EXPLAINED There are nine questions to ask when conducting multicenter database research (Table 1). Three of these are discussed in detail here using reports from the PHIS, VPS, and ELSO datasets to illustrate what is meant. Of course, the same considerations apply to all dataset reports seeking publication in PCCM. TABLE 1. - Checklist for Database Research Intended For Pediatric Critical Care Medicine Nine Questions to Consider 1. What is the new pediatric critical care research question and hypothesis? 2. Have the local or national institutional review board requirements been met? 3. Has an in-depth literature review of prior publications using the database/registry been completed? 4. What are the patient inclusions and exclusions, and what are the primary and secondary PICU outcomes? 5. Have the best variables for the pediatric critical care research question been considered? 6. Are you sure that the definitions of these variables in the database/registry have not changed over time? 7. Are there any known associations that arise from your literature review (Question No. 3) that mean these variables should also be included in any new outcome model? 8. What is the analytical strategy for missing data? 9. Can you provide an “at the bedside” or “what this study means” conclusion to your work? The Research Idea (Question No. 1) There are several research options when preparing to use a curated dataset. One may be as straightforward as descriptive epidemiology of PICU practice. For example, the PHIS dataset often features as a starting point for a new field of research. In 2022, a group of investigators reported the scale of PICU admissions based on the 2014−2019 PHIS dataset from 43 U.S. children’s hospitals (8). In this 6-year period, data from 623,511 PICU admissions (excluding newborns and those > 64 yr) provided information about PICU resource allocation and capacity over time. The investigators then looked at a subset of the PHIS data to inform ideas about a population with two or more subsequent admissions within a year of an index admission (9). These high-frequency admissions accounted for around 6% of the 2018 PICU admissions in the PHIS data (now using 47 U.S. children’s hospitals and restricted to those aged ≤ 18 yr). The same investigators then turned their attention to the 2017−2019 VPS dataset from 131 U.S. PICUs and selected 291,583 admissions aged < 21 years (10). Here, they tested various definitions of medical complexity, which was not possible in the PHIS data. A different option used in some dataset research is to examine public health relationships. For example, outcomes associated with regional access to healthcare or social determinants of health (11–14). Alternatively, what about characteristics and outcomes of a practice, diagnosis or disease and the treatments? For example: the practice of airway management (15,16), tracheostomy (17–21), and cardiac arrest needing ECMO (22–24); the diagnosis of suicide (25,26), critical bronchiolitis (27–29) or critical asthma (29–31); and hematology, oncology, and immunologic disease in the PICU (32–35). Each of these research activities move beyond one’s local practice to the national or international level. Last, how about an improved understanding of population survival physiology. Here, consider the exploratory hypotheses about organ systems and the brain in relation to arterial partial pressure of oxygen (PaO2) or carbon dioxide (PaCO2), and blood pressure (24,36–38). The Review of Prior Research (Question No. 3) The research opportunities provided by datasets and registries are immense because they can support multiple reports with different hypotheses in distinct populations. Even so, prior research means there is risk of overlap or loss of uniqueness in a research effort. Therefore, do consider the overlap between previously used and currently chosen variables, and what is unique in a planned study. Table 2 summarizes the PHIS, VPS, and ELSO research already mentioned. The details show how carefully PCCM’s author-investigators have used the respective strengths of each dataset to follow through a theme of research. This information can also be used to guide further investigations. For example, consider the three bronchiolitis reports (27–29). There are two non-overlapping VPS reports. The first study covered the period 2013−2017 with a population of over 14,000 from 128 PICUs in the United States (27). The investigators developed a Critical Bronchiolitis Score (based on acute physiologic changes in acid base, vital signs, and clinical trajectory) and examined its association with use of PICU-level respiratory support and length of stay. The second VPS study covered another bronchiolitis population of over 14,000 managed in 81 PICUs, 2018−2020 (29). Here, the investigators looked at a 20-week period in 2020, referenced to 2018−2019 data, to assess the relationship between COVID-19-related school closures and expected PICU admission numbers. The other bronchiolitis report uses the PHIS dataset spanning the 2009−2019 experience in 38 children’s hospitals (28). Even though there is some overlap in the timing of these data with the VPS datasets, the report is a unique research output. The investigators presented the children’s hospitals 283,006 bronchiolitis admissions, and the charges in the subgroup of 75,020 needing PICU admission. Given this background, what are the next iterations or interventions in bronchiolitis care that we should focus on using the curated datasets? TABLE 2. - Comparative Review of Pediatric Health Information System, Virtual Pediatric Systems, LLC, and Extracorporeal Life Support Organization Reports in Pediatric Critical Care Medicine, 2022−2024 Topic Dataset (Reference) Years Population Subset Age Subset Outcomes Epidemiology PHIS (8) 2014-2019 623,511 (43 CH-PIC) 623,511 > nb–< 65 y Costs, LOS Epidemiology PHIS (9) 2018 95,465 (47 CH-PIC) 5,880 ≤ 18 y HFPICU admit Epidemiology VPS (10) 2017-2019 291,583 (131 PICUs) 226,430 < 21 y Complexity SDOH PHIS (11) 2017-2019 1,193 (38 CH-PIC) 143 > 30 d IPCC during SCT SDOH PHIS (12) 2018-2019 78,839 (43 CH-PIC) 9,955 < 18 y Readmits by COI SDOH VPS (13) 2019-2020 33,901 (15 PICUs) N/A < 18 y Mortality by COI SDOH PHIS (14) 2019 178,134 (45 CH-PIC) 44,200 < 18 y PICU admits Airway VPS (15) 2009-2018 69,739 (79 cardiac ICUs) 52,804 < 18 y <6 h ex-T Airway VPS (16) 2012-2020 5,703 (149 PICUs) 1,661 ≤ 18 y Re-T in U-ex-T Tracheostomy ELSO (17) 2015-2019 3,685 (>7d ECMO) 94 <18 y TT-ECMO, LOS Tracheostomy PHIS (18) 2009-2017 793 (37 CH-PIC) 793 31 d–21 y TT+PH mortality Tracheostomy PHIS (19) 2010-2020 16,121 (52 CH-PIC) 10,295 < 18 y TT early timing Tracheostomy PHIS (20) 2004-2019 1,645 (49 CH-PIC) 251 < 90 d TT/GT in TA Tracheostomy PHIS (21) 2009-2020 1061 (48 CH-PIC) 217 0–21 y TT and HSCT E-CPR VPS (22) 2010-2018 12,931 (CA PICUs) 650 < 18 y E-CPR mortality E-CPR ELSO (23) 2017-2019 567 (noncardiac) 567 <18 y E-CPR mortality E-CPR ELSO (24) 2017-2021 2,209 138 ≥ 29 d–< 18 y E-CPR DNC Suicide VPS (25) 2009-2017 9,187 (137 PICUs) 9,187 6–18 y Epidemiology Suicide VPS (26) 2016-2021 7,692 (69 PICUs) 7,692 > 12 y–< 18 y Pre/Post-COVID Bronchiolitis VPS (27) 2013-2017 14,407 (128 PICUs) 3,481 < 2 y Early MV score Bronchiolitis PHIS (28) 2009-2019 283,006 (38 CH-PIC) 283,006 < 2 y LOS and costs Bronchiolitis VPS (29) 2018-2020 14,129 (81 PICUs) 14,129 Age N/A Admits in COVID Asthma VPS (29) 2018-2020 6,588 (81 PICUs) 6,588 Age N/A Admits in COVID Asthma VPS (30) 2010-2020 221 (61 PICUs) 221 < 18 y ECMO/IA Asthma PHIS (31) 2013-2021 213,506 (39 CH-PIC) 29,026 2–18 y PICU admits Heme/Onc ELSO (32) 1989-2018 207 (immune Δs) 207 1 m–18 y ECMO survival Heme/Onc ELSO (33) 2000-2019 902 (neoplasm) 902 ≤ 18 y ECMO survival Heme/Onc VPS (34) 2020-2021 55 (21 PICUs) 55 ≥ 29 d ALL and CRS Heme/Onc PHIS (35) 2012-2021 55,827 (36 CH-PIC) 55,827 ≤ 18 y Cancer patients Physiology ELSO (24) 2017-2021 138 E-CPR 138 ≥ 29 d–< 18 y PaCO2 outcome Physiology VPS (36) 2015-2019 13,071 (136 PICUs) 13,071 ≤ 18 y PaO2 outcome Physiology ELSO (37) 2010-2019 7,270 (PaCO2/MAP) 1,131 > 30 d–< 18 y CNS outcomes Physiology ELSO (38) 2015-2020 3,533 (PaO2) 3,533 ≤ 28 d 28-day mortality ALL = acute lymphoblastic leukemia, CA = cardiac arrest, CH-PIC = children’s hospital pediatric intensive care, COI = child opportunity index, CRS = cytokine release syndrome, d, day, Δs = diseases, DNC, death using neurologic criteria, E-CPR = ECMO support during cardiopulmonary resuscitation, ex-T = extubation, Heme/Onc = hematology/oncology, HFPICU = high-frequency PICU, IA = inhalational anesthesia, IPCC = inpatient palliative care consultation, LOS = length of stay, MAP = mean arterial pressure, m = month, MV = mechanical ventilation, nb = newborn, N/A = not available, PaCO2 = arterial partial pressure of carbon dioxide, PaO2 = arterial partial pressure of oxygen, Re-T = re-intubation, SCT = stem cell transplant, SDOH = social determinants of health, TA = truncus arteriosus, TT = tracheostomy, TT-ECMO = tracheostomy during ECMO, TT/GT = TT and/or gastrostomy, TT-PH = tracheostomy for pulmonary hypertension, U-ex-T = unplanned extubation, y = year. The Missing Data Strategy (Question No. 8) Missing data is a problem when managing large datasets and there is no practical way of searching the original record for lost information. One approach is to remove cases with missing observations. Alternatively, normal values can be substituted for missing information. A modeling report in 2022 tested these approaches in the VPS 2017 dataset (from 117 PICUs) in 123,035 unique patient episodes (39). The investigators showed that multiple imputations using chained equations outperformed discarding cases or assuming missing values. Whatever the solution, PCCM recommends seeking statistical advice and expertise (4). CONCLUSION PCCM welcomes epidemiological research using large, multicenter, curated datasets. This Writing for PCCM article has focused on using the PHIS, VPS, and ELSO datasets, but the principles outlined apply to any other dataset being reported in PCCM. The Checklist presented in Table 1 is intended to help investigators at a preliminary stage of their work. It can also be used by reviewers and readers. Formulate an opinion about collections of reports (Table 2). Question the uniqueness of new variables in a new study. Include in the analyses variables previously identified as being associated with the outcome described in the new report. Finally, at the time of writing, refer to the Strengthening The Reporting of OBservational studies in Epidemiology (STROBE) statement and follow the recommendations (2,40).

Topics & Concepts