Can Chat-GPT read and understand guidelines? An example using the S2k guideline intrauterine growth restriction of the German Society for Gynecology and Obstetrics
Iason Psilopatis, Simon Bader, Annika Krueckel, Sven Kehl, Matthias W. Beckmann, Julius Emons
Abstract
PURPOSE: To investigate the capacity of chat-generative pre-trained transformer (Chat-GPT) to understand the S2k guideline of the German Society for Gynecology and Obstetrics on intrauterine growth restriction. METHODS: The German-language free Chat-GPT version was used to test the ability of Chat-GPT to understand the definition of small for gestational age and intrauterine growth restriction, to indicate the correct time and place of delivery and to evaluate ist ability to recommend a spontaneous delivery versus a primary caesarean section in accordance with the guideline recommendations. In order to objectively evaluate the suggestions a simple three-color 'traffic light' evaluation system was employed. RESULTS: Almost all Chat-GPT's suggestions in the context of definition of small for gestational age/intrauterine growth restriction as well as correct time of delivery were adequate, whereas more than half of the suggestions made in terms of correct delivery mode needed reformulation or even correction. CONCLUSION: Chat-GPT appears to be a valuable form of artificial intelligence that could be integrated into everyday clinical practice.