A video-based situational judgement test of medical students’ communication competence in patient encounters: Development and first evaluation
Sabine Reiser, Laura Schacht, Eva Thomm, Christina Figalist, Laura Janssen, Kristina Schick, Eva Dörfler, Pascal O. Berberat, Martin Gartmeier, Johannes Bauer
Abstract
OBJECTIVE: We developed and evaluated the Video-Based Assessment of Medical Communication Competence (VA-MeCo), a construct-driven situational judgement test measuring medical students' communication competence in patient encounters. METHODS: = 13) to ensure curricular and content validity and sufficient expert agreement on the answer key. In the evaluation phase, we conducted a cognitive pre-test (n = 12) and a pilot study (n = 117) with medical students to evaluate test usability and acceptance, item statistics and test reliability depending on the applied scoring method (raw consensus vs. pairwise comparison scoring). RESULTS: The results of the expert interviews indicated good curricular and content validity. Expert agreement on the answer key was high (ICCs> .86). The pilot study showed favourable usability and acceptance by students. Irrespective of the scoring method, reliability for the complete test (Cronbach's α >.93) and its subscales (α >.83) was high. CONCLUSION: There is promising evidence that medical communication competence can be validly and reliably measured using a construct-driven and video-based situational judgement test. PRACTICE IMPLICATIONS: Video-based SJTs allow efficient online assessment of medical communication competence and are well accepted by students and educators.