AgentMD: Empowering language agents for risk prediction with large-scale clinical tool learning
Qiao Jin, Zhizheng Wang, Yifan Yang, Qingqing Zhu, D. Wright, Thomas Huang, Nikhil Khandekar, Nicholas Wan, Xuguang Ai, W. John Wilbur, Zhe He, Richard A. Taylor, Qingyu Chen, Zhiyong Lu
Abstract
Clinical calculators play a vital role in healthcare, but their utilization is often hindered by usability and dissemination challenges. We introduce AgentMD, a novel language agent capable of curating and applying clinical calculators across various clinical contexts. As a tool builder, AgentMD first uses PubMed to curate a diverse set of 2,164 executable clinical calculators with over 85% accuracy for quality checks and over 90% pass rate for unit tests. As a tool user, AgentMD autonomously selects and applies the relevant clinical calculators. Our evaluations show that AgentMD significantly outperforms GPT-4 for risk prediction (87.7% vs. 40.9% in accuracy). Results on 698 real-world emergency department notes confirm that AgentMD accurately computes medical risks at an individual level. Moreover, AgentMD can provide population-level insights for institutional risk management. Our study illustrates the capabilities of language agents to curate and utilize clinical calculators for both individual patient care and at-scale healthcare analytics.