A foundation model for human-AI collaboration in medical literature mining
Zifeng Wang, Lang Cao, Qiao Jin, Joey Wing Yan Chan, Nicholas Wan, Behdad Afzali, Hyun-Jin Cho, C. Y. Choi, Mehdi Emamverdi, Manjot K. Gill, Sunhyung Kim, Yijia Li, Yi Liu, Yiming Luo, Hanley Ong, Justin F. Rousseau, Irfan Sheikh, Jenny J. Wei, Ziyang Xu, Christopher M. Zallek, Kyungsang Kim, Yifan Peng, Zhiyong Lu, Jimeng Sun
Abstract
Applying artificial intelligence (AI) for systematic literature review holds great potential for enhancing evidence-based medicine, yet has been limited by insufficient training and evaluation. Here, we present LEADS, an AI foundation model trained on 633,759 samples curated from 21,335 systematic reviews, 453,625 clinical trial publications, and 27,015 clinical trial registries. In experiments, LEADS demonstrates consistent improvements over four cutting-edge large language models (LLMs) on six literature mining tasks, e.g., study search, screening, and data extraction. We conduct a user study with 16 clinicians and researchers from 14 institutions to assess the utility of LEADS integrated into the expert workflow. In study selection, experts using LEADS achieve 0.81 recall vs. 0.78 without, saving 20.8% time. For data extraction, accuracy reached 0.85 vs. 0.80, with 26.9% time savings. These findings encourage future work on leveraging high-quality domain data to build specialized LLMs that outperform generic models and enhance expert productivity in literature mining.