Chatgpt vs traditional pedagogy: a comparative study in urological learning
Alessio Digiacomo, Angelo Orsini, Rossella Cicchetti, Ludovica Spadano, Sara De Santis, Luca Sessa, Miriana Vitale, Marta Di Nicola, Flavia Tamborino, Martina Basconi, Riccardo De Archangelis, Gaetano Salzano, Guglielmo Dello Stritto, Peppino Lannutti, Luigi Schips, Michele Marchioni
Abstract
PURPOSE: Technological evolution is radically changing medical learning models. We evaluated the learning outcomes of urological concepts using ChatGPT, traditional lecture and combined approach. METHODS: We conducted a randomized triple-blind study on 121 medical students with no previous formal curriculum in urology. Students were randomly divided into three study classes with different learning methods: ChatGPT, Lecture and ChatGPT + Lecture. The "adrenal glands" were randomly extracted as the subject of the lessons. Students were evaluated using a thirty-question test. RESULTS: The evaluation test median score was higher for students who underwent ChatGPT + Lecture compared with those who had only ChatGPT (10 vs. 12, p = 0.007). Such differences remained statistically significant also in multivariable models adjusting according to year of course, gender and previous ChatGPT experience (estimate: 2.6, p-value = 0.002). For most of the questions (about 70%), the proportion of students correctly answering was higher in the ChatGPT + Lecture learning groups than in the other groups. CONCLUSION: ChatGPT loses its potential if used without a previous background. The limits of scientific reliability persist and a teacher-guided method is still essential. ChatGPT + traditional lecture gives more effective results than the single traditional lecture also allowing a better use of the chatbot.