Text summarization with ChatGPT for drug labeling documents
Lan Ying, Zhichao Liu, Hong Fang, Rebecca Kusko, Leihong Wu, Stephen Harris, Weida Tong
Abstract
Text summarization is crucial in scientific research, drug discovery and development, regulatory review, and more. This task demands domain expertise, language proficiency, semantic prowess, and conceptual skill. The recent advent of large language models (LLMs), such as ChatGPT, offers unprecedented opportunities to automate this process. We compared ChatGPT-generated summaries with those produced by human experts using FDA drug labeling documents. The labeling contains summaries of key labeling sections, making them an ideal human benchmark to evaluate ChatGPT’s summarization capabilities. Analyzing >14 000 summaries, we observed that ChatGPT-generated summaries closely resembled those generated by human experts. Importantly, ChatGPT exhibited even greater similarity when summarizing drug safety information. These findings highlight ChatGPT’s potential to accelerate work in critical areas, including drug safety. Teaser ChatGPT reliably generates summaries closely resembling those of human experts, particularly in the context of summarizing drug safety information using FDA labeling as a benchmark.