OpenAI's GPT‐4 performs to a high degree on board‐style dermatology questions
Marcus L. Elias, Joshua Burshtein, Victoria R. Sharon
Abstract
BACKGROUND: Artificial intelligence tools such as OpenAI's GPT-4 have shown promise in medical education, but their potential in dermatology remains unexplored. OBJECTIVES: To assess GPT-4's performance on dermatology board-style questions and determine its value as a supplementary educational tool for trainees and educators. METHODS: This cross-sectional study evaluated GPT-4's performance on 250 random dermatology board-style questions sampled from the American Academy of Dermatology's Board Prep Plus resource. Questions were divided into five subspecialties and various difficulty levels. GPT-4 responses were compared to the correct answers and evaluated by two physicians. RESULTS: GPT-4 achieved an overall accuracy of 75% on the 250 questions, with no significant variation based on subspecialty or question difficulty. The most common errors were factual and misunderstanding inaccuracies. Responses scored high in clarity, accuracy, and relevance but frequently lacked depth and completeness. CONCLUSION: GPT-4 performed to a high degree and demonstrated promising performance as an educational adjunct in dermatology. Improvements in response depth and completeness are needed before its use as an unsupervised learning tool is established.