Pediatric dermatologists versus <scp>AI</scp> bots: Evaluating the medical knowledge and diagnostic capabilities of <scp>ChatGPT</scp>
Charles Huang, Esther Zhang, Marie‐Chantal Caussade, Trinity Brown, Griffin Stockton Hogrogian, Albert C. Yan
Abstract
This study evaluates the clinical accuracy of OpenAI's ChatGPT in pediatric dermatology by comparing its responses on multiple-choice and case-based questions to those of pediatric dermatologists. ChatGPT's versions 3.5 and 4.0 were tested against questions from the American Board of Dermatology and the "Photoquiz" section of Pediatric Dermatology. Results show that human pediatric dermatology clinicians generally outperformed both ChatGPT iterations, though ChatGPT-4.0 demonstrated comparable performance in some areas. The study highlights the potential of AI tools in aiding clinicians with medical knowledge and decision-making, while also emphasizing the need for continual advancements and clinician oversight in using such technologies.