LINS: A general medical Q&A framework for enhancing the quality and credibility of LLM-generated responses
Sheng Wang, Fangyuan Zhao, Dan Bu, Yang Lu, Ming Gong, Hongjie Liu, Zhaohui Yang, Xiaoxi Zeng, Zhiyuan Yuan, Zhiyuan Yuan, Jingbo Sun, Yang Wu, Lianhe Zhao, Xirun Wan, Wei Huang, Tao Wang, Mengtong Xu, Jianjun Luo, Jingjia Liu, Jianjun Zheng, Wei Zhang, Kang Zhang, Hongjia Zhang, Shu Wang, Runsheng Chen, Yi Zhao, Runsheng Chen, Yi Zhao
Abstract
Large language models can lighten the workload of clinicians and patients, yet their responses often include fabricated evidence, outdated knowledge, and insufficient medical specificity. We introduce a general retrieval-augmented question-answering framework that continuously gathers up-to-date, high-quality medical knowledge and generates evidence-traceable responses. Here we show that this approach significantly improves the evidence validity, medical expertise, and timeliness of large language model outputs, thereby enhancing their overall quality and credibility. Evaluation against 15,530 objective questions, together with two physician-curated clinical test sets covering evidence-based medical practice and medical order explanation, confirms the improvements. In blinded trials, resident physicians indicate meaningful assistance in 87.00% of evidence-based medical scenarios, and lay users find it helpful in 90.09% of medical order explanations. These findings demonstrate a practical route to trustworthy, general-purpose language assistants for clinical applications. To fully realize LLMs’ potential value in clinical applications, effective methods to enhance their quality and credibility are required. Here, the authors present LINS, a framework to enhance medical LLM responses by integrating up-to-date evidence and supporting clinical tasks, and validate it through new physician-curated datasets and large-scale user trials.