To what extent does ChatGPT understand genetics?
Teymoor Khosravi, Zainab M. Al Sudani, Morteza Oladnabi
Abstract
OpenAI’s ChatGPT, is a conversational chatbot that uses Generative Pre-trained Transformer or GPT language model to mimic human-like responses. Here we evaluated its performance in providing responses to genetics questions across five different tasks including solid genetic basics, identifying inheritance pattern based on described pedigrees, interpreting genetic mutation notations, solving genetic population problems, and taking a medical genetics Ph.D. entrance exam. Our results showed that ChatGPT was able to generate correct answers to approximately 70% of questions (n = 145). Its performance on descriptive and memorisation tasks showed more accuracy compared to analytical and critical thinking ones. Failure to capture human writing nuances in questions, and applying genetic basics to solve problems, alongside providing false information were the most notable drawbacks. However, overall results were promising suggesting that ChatGPT could be a well-prepared assistant for genetic educators and healthcare providers.