Simulated diagnostic performance of low-field MRI: Harnessing open-access datasets to evaluate novel devices
Thomas Arnold, Steven N. Baldassano, Brian Litt, Joel M. Stein
Abstract
The purpose of this study is to demonstrate a method for virtually evaluating novel imaging devices using machine learning and open-access datasets, here applied to a new, low-field strength portable 64mT MRI device. Paired 3 T and 64mT brain images were used to develop and validate a transformation converting standard clinical images to low-field quality images. Separately, 3 T images were aggregated from open-source databases spanning four neuropathologies: low-grade glioma (LGG, N = 76), high-grade glioma (HGG, N = 259), stroke (N = 28), and multiple sclerosis (MS, N = 20). The transformation method was then applied to the open-source data to generate simulated low-field images for each pathology. Convolutional neural networks (DenseNet-121) were trained to detect pathology in axial slices from either 3 T or simulated 64 mT images, and their relative performance was compared to characterize the potential diagnostic capabilities of low-field imaging. Algorithm performance was measured using area under the receiver operating characteristic curve. Across all cohorts, pathology detection was similar between 3 T and simulated 64mT images (LGG: 0.97 vs. 0.98; HGG: 0.96 vs. 0.95; stroke: 0.94 vs. 0.94; MS: 0.90 vs 0.87). Pathology detection was further characterized as a function of lesion size, intensity, and contrast. Simulated images showed decreasing sensitivity for lesions smaller than 4 cm2. While simulations cannot replace prospective trials during the evaluation of medical devices, they can provide guidance and justification for prospective studies. Simulated data derived from open-source imaging databases may facilitate testing and validation of new imaging devices.