ChatGPT as a Source of Information for Bariatric Surgery Patients: a Comparative Analysis of Accuracy and Comprehensiveness Between GPT-4 and GPT-3.5
Jamil S. Samaan, Nithya Rajeev, Wee Han Ng, Nitin Srinivasan, Jonathan A. Busam, Yee Hui Yeo, Kamran Samakar
Abstract
Bariatric surgery is an effective and safe treatment for severe obesity [ 1 ]. Accurate and comprehensive perioperative education is integral to patients’ surgical journeys and outcomes. Large language models (LLMs), like ChatGPT, have the potential to revolutionize patient education by leveraging vast quantities of data to respond to user prompts in an easy-to-understand and conversational manner. Released by OpenAI in November of 2022, GPT-3.5 acquired 1 million users within 5 days of its release, outpacing applications such as Facebook, Twitter, and Instagram [ 2 ]. By January of 2023, its user base reached 100 million monthly active users, making it the fastest growing consumer application in history [ 3 ]. Our recent study demonstrated the impressive ability of GPT-3.5 in answering questions related to bariatric surgery, showing high accuracy, comprehensiveness, and reproducibility of responses [ 4 ]. GPT-3.5’s successor, GPT-4, was released in March of 2023 with improvements in performance across multiple domains [ 5 , 6 , 7 , 8 ]. The current study builds on our previous analysis by comparing the accuracy and comprehensiveness of GPT-4 compared to GPT-3.5, in answering questions related to bariatric surgery.