When vision meets reality: Exploring the clinical applicability of GPT-4 with vision
Jiawen Deng, Kiyan Heybati, Matthew Shammas‐Toma
Abstract
•GPT-4V is a newly introduced large language model that can interpret images and discuss its findings conversationally. •We used GPT-4V to interpret 10 radiological images and found that it had limited overall accuracy and precision. •GPT-4V's performance was inconsistent both between images and within multiple interpretations of the same image. •GPT-4V has a tendency to hallucinate. This is worsened by misleading prompts, file names, on-screen texts and markers. •While unsuitable for image interpretation, GPT-4V may be useful for translating and summarizing radiology reports.
Topics & Concepts
TransformerMedicineArtificial intelligenceArchitectureGenerative grammarThe InternetComputer visionNatural language processingComputer scienceWorld Wide WebElectrical engineeringEngineeringVisual artsVoltageArtArtificial Intelligence in Healthcare and EducationMachine Learning in HealthcareTopic Modeling