Validation of an Instrument for Real-Time Assessment of Neonatal Intubation Skills: A Randomized Controlled Simulation Study
Lindie J.M.K. Kuijpers, Mathijs Binkhorst, Nicole K. Yamada, Romy N. Bouwmeester, Arno van Heijst, Louis P. Halamek, Marije Hogeveen
Abstract
OBJECTIVE: This study aimed to evaluate the construct validity and reliability of real-time assessment of a previously developed neonatal intubation scoring instrument (NISI). STUDY DESIGN: We performed a randomized controlled simulation study at a simulation-based research and training facility. Twenty-four clinicians experienced in neonatal intubation ("experts") and 11 medical students ("novices") performed two identical elective intubations on a neonatal patient simulator. Subjects were randomly assigned to either the intervention group, receiving predefined feedback between the two intubations, or the control group, receiving no feedback. Using the previously developed NISI, all intubations were assessed, both in real time and remotely on video. Construct validity was evaluated by (1) comparing the intubation performances, expressed as percentage scores, with and without feedback, and (2) correlating the intubation performances with the subjects' level of experience. The intrarater reliability, expressed as intraclass correlation coefficient (ICC), of real-time assessment compared with video-based assessment was determined. RESULTS: 0.01). ICC's were 0.95 (95% CI: 0.89-0.97) and 0.94 (95% CI: 0.89-0.97) for the first and second intubation, respectively. CONCLUSION: Our NISI has construct validity and is reliable for real-time assessment. KEY POINTS: · Our neonatal intubation scoring instrument has construct validity.. · Our instrument can be reliably employed to assess neonatal intubation skills directly in real time.. · It is suitable for formative assessment, i.e., providing direct feedback during procedural training..