How Good Is ChatGPT for Medication Evidence Synthesis?
Hao Liu, Yifan Peng, Chunhua Weng
Abstract
With its seeming competence to mimic human responses, ChatGPT, an emerging AI-powered chatbot, has spurred great interest. This study aims to explore the role of ChatGPT in synthesizing medication literature and compare it with a hybrid summarization system. We tested ten medications' effectiveness with reference to their definitions and descriptions extracted from DrugBank. ChatGPT could generate coherent summaries that are not backed by evidence. In contrast, our approach can provide a highly structured and concise synthesis of related evidence, but the resulting summary is not as fluent and convincing as ChatGPT. Therefore, we recommend integrating both techniques to achieve the best performance.
Topics & Concepts
ChatbotAutomatic summarizationComputer scienceDrugBankCompetence (human resources)Data scienceArtificial intelligenceNatural language processingPsychologySocial psychologyDrugPsychiatryTopic ModelingMachine Learning in HealthcareArtificial Intelligence in Healthcare and Education