Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks
Jack Gallifant, Shan Chen, Pedro Moreira, Nikolaj Munch, Mingye Gao, Jackson Pond, Leo Anthony Celi, Hugo J.W.L. Aerts, Thomas Hartvigsen, Danielle S. Bitterman
Abstract
, to evaluate performance differences on medical benchmarks after swapping brand and generic drug names using physician expert annotations. We assess both open-source and API-based LLMs on MedQA and MedMCQA, revealing a consistent performance drop ranging from 1-10%. Furthermore, we identify a potential source of this fragility as the contamination of test data in widely used pre-training datasets.
Topics & Concepts
Computer scienceDrugNatural language processingLinguisticsMedicinePharmacologyPhilosophyGenomics and Rare DiseasesBiomedical Text Mining and Ontologies