Capacity for large language model chatbots to aid in orthopedic management, research, and patient queries
Branden Sosa, Michelle Cung, Vincentius J. Suhardi, Kyle W. Morse, Andrew Thomson, He S. Yang, Sravisht Iyer, Matthew B. Greenblatt
Abstract
Large language model (LLM) chatbots possess a remarkable capacity to synthesize complex information into concise, digestible summaries across a wide range of orthopedic subject matter. As LLM chatbots become widely available they will serve as a powerful, accessible resource that patients, clinicians, and researchers may reference to obtain information about orthopedic science and clinical management. Here, we examined the performance of three well-known and easily accessible chatbots-ChatGPT, Bard, and Bing AI-in responding to inquiries relating to clinical management and orthopedic concepts. Although all three chatbots were found to be capable of generating relevant responses, ChatGPT outperformed Bard and BingAI in each category due to its ability to provide accurate and complete responses to orthopedic queries. Despite their promising applications in clinical management, shortcomings observed included incomplete responses, lack of context, and outdated information. Nonetheless, the ability for these LLM chatbots to address these inquires has largely yet to be evaluated and will be critical for understanding the risks and opportunities of LLM chatbots in orthopedics.