Head-to-head comparison of commercial artificial intelligence solutions for detection of large vessel occlusion at a comprehensive stroke center
Jacob Schlossman, Daniel Ro, Shirin Salehi, Daniel Chow, Wengui Yu, Peter Chang, Jennifer E. Soun
Abstract
Purpose: Despite the availability of commercial artificial intelligence (AI) tools for large vessel occlusion (LVO) detection, there is paucity of data comparing traditional machine learning and deep learning solutions in a real-world setting. The purpose of this study is to compare and validate the performance of two AI-based tools (RAPID LVO and CINA LVO) for LVO detection. Materials and methods: = 263) for suspected stroke were evaluated for LVO. RAPID LVO is a traditional machine learning model which primarily relies on vessel density threshold assessment, while CINA LVO is an end-to-end deep learning tool implemented with multiple neural networks for detection and localization tasks. Reasons for errors were also recorded. Results: There were 29 positive and 224 negative LVO cases by ground truth assessment. RAPID LVO demonstrated an accuracy of 0.86, sensitivity of 0.90, specificity of 0.86, positive predictive value of 0.45, and negative predictive value of 0.98, while CINA demonstrated an accuracy of 0.96, sensitivity of 0.76, specificity of 0.98, positive predictive value of 0.85, and negative predictive value of 0.97. Conclusion: Both tools successfully detected most anterior circulation occlusions. RAPID LVO had higher sensitivity while CINA LVO had higher accuracy and specificity. Interestingly, both tools were able to detect some, but not all M2 MCA occlusions. This is the first study to compare traditional and deep learning LVO tools in the clinical setting.