Performance of an Artificial Intelligence System for Breast Cancer Detection on Screening Mammograms from BreastScreen Norway
Marthe Larsen, Camilla F. Olstad, Christoph I. Lee, Tone Hovda, Solveig Roth Hoff, Marit Almenning Martiniussen, Karl Øyvind Mikalsen, Håkon Lund-Hanssen, Helene S. Solli, Marko Silberhorn, Åse Ø Sulheim, Steinar Auensen, Jan F. Nygård, Solveig Hofvind
Abstract
“Just Accepted” papers have undergone full peer review and have been accepted for publication in Radiology: Artificial Intelligence. This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content. Purpose To explore the standalone breast cancer detection performance at different risk score thresholds of a commercially available artificial intelligence (AI) system. Materials and Methods This retrospective study included information from 661,695 digital mammographic examinations performed among 242,629 female individuals screened as a part of x, 2004–2018. The study sample included 3807 screen-detected cancers (SDC) and 1110 interval breast cancers (IC). A continuous examination level risk score by the AI system was used to measure performance as the area under the receiver operating characteristic curve (AUC) with 95% CIs and cancer detection at different AI risk score thresholds. Results The AUC of the AI system was 0.93 (95% CI: 0.92–0.93) for SDC and IC combined and 0.97 (95% CI: 0.97–0.97) for SDC. In a setting where 10% of the examinations with the highest AI risk scores were defined as positive and 90% with the lowest scores as negative, 92.0% (3502/3807) of the SDC and 44.6% (495/1100) of the IC were identified by AI. In this scenario, 68.5% (10 987/16 029) of false positive screening results (negative recall assessment) were considered negative by AI. When 50% was used as the cut-off, 99.3% (3781/3807) of the SDC and 85.2% (946/1100) of the IC were identified as positive by AI, while 17.0% (2725/16 029) of the false positives were considered as negative. Conclusion The AI system showed high performance in detecting breast cancers within 2 years of screening mammography and a potential for triaging low-risk mammograms to reduce radiologist workload. ©RSNA, 2024