Large Language Models Are Poor Medical Coders — Benchmarking of Medical Code Querying
Ali Soroush, Benjamin S. Glicksberg, Eyal Zimlichman, Yiftach Barash, Robert Freeman, Alexander W. Charney, Girish N. Nadkarni, Eyal Klang
Abstract
BACKGROUND Large language models (LLMs) have attracted significant interest for automated clinical coding. However, early data show that LLMs are highly error-prone when mapping medical codes. We sought to quantify and benchmark LLM medical code querying errors across several available LLMs.
Topics & Concepts
BenchmarkingComputer scienceCode (set theory)Diagnosis codeNatural language processingProgramming languageMedicineBusinessSet (abstract data type)MarketingPopulationEnvironmental healthBiomedical Text Mining and OntologiesArtificial Intelligence in HealthcareArtificial Intelligence in Healthcare and Education