LLM-powered breast cancer staging from PET/CT reports: a comparative performance study
Daniel Spitzl, Markus Mergen, Rickmer Braren, Lukas Endrös, Matthias Eiber, Lisa Steinhelfer
Abstract
• We assessed LLMs’ ability to assign TNM stages from breast PET/CT reports. • Four models analyzed 111 synthetic reports following UICC/AJCC breast guidelines. • Claude 3.5 Sonnet achieved top F1 scores: 0.95 T, 0.95 N, 1.00 M, 0.92 stage. • LLM-based staging shows promise for streamlining multidisciplinary care decisions. • Prospective trials must confirm performance before clinical adoption is pursued. Imaging reports are crucial in breast cancer management, with the tumor-node-metastasis (TNM) classification serving as a widely used model for assessing disease severity, guiding treatment decisions, and predicting patient outcomes. Large language models (LLMs) offer a potential solution by extracting standardized UICC TNM classifications and the corresponding UICC stage directly from existing PET/CT reports. This approach holds promise to enhance staging accuracy, streamline multidisciplinary discussions, and improve patient outcomes. Here, we evaluated four LLMs—ChatGPT-4o, DeepSeek V3, Claude 3.5 Sonnet, and Gemini 2.0 Flash—for their capacity to determine TNM staging based on UICC/AJCC breast cancer guidelines. A total of 111 fictitious PET/CT reports were analyzed, and each model’s outputs were measured against expert-generated TNM classifications and stage categorizations. Among the tested models, Claude 3.5 Sonnet demonstrated superior F1 scores of 0.95%, 0.95%, 1.00% and 0.92% for T, N, M classification and UICC stage classification, respectively. These findings underscore the ability of advanced natural language processing (NLP) technologies to support reliable cancer staging, potentially aiding clinicians. Despite the encouraging performance, prospective clinical trials and validation across diverse practice settings remain critical to confirming these preliminary outcomes. Nonetheless, this study highlights the promise of LLM-based systems in reinforcing the accuracy of oncologic workflows and lays the groundwork for broader adoption of AI-driven tools in breast cancer management.