Longitudinal deep neural networks for assessing metastatic brain cancer on a large open benchmark
Katherine E. Link, Zane Schnurman, Chris Liu, Young Joon Kwon, Lavender Yao Jiang, Mustafa Nasir-Moin, Sean N. Neifert, Juan Diego Alzate, Kenneth Bernstein, Tanxia Qu, Viola Chen, Eunice Yang, John G. Golfinos, Daniel A. Orringer, Douglas Kondziolka, Eric K. Oermann
Abstract
The detection and tracking of metastatic cancer over the lifetime of a patient remains a major challenge in clinical trials and real-world care. Advances in deep learning combined with massive datasets may enable the development of tools that can address this challenge. We present NYUMets-Brain, the world’s largest, longitudinal, real-world dataset of cancer consisting of the imaging, clinical follow-up, and medical management of 1,429 patients. Using this dataset we developed Segmentation-Through-Time, a deep neural network which explicitly utilizes the longitudinal structure of the data and obtained state-of-the-art results at small (<10 mm3) metastases detection and segmentation. We also demonstrate that the monthly rate of change of brain metastases over time are strongly predictive of overall survival (HR 1.27, 95%CI 1.18-1.38). We are releasing the dataset, codebase, and model weights for other cancer researchers to build upon these results and to serve as a public benchmark. Cancer is a dynamic disease, with one of its deadly complications being metastatic brain tumors. Here, the authors present a large, multimodal, longitudinal dataset of metastatic cancer, assembled from real world data for cancer research and artificial intelligence (AI) model development. They train time-dependent AI models, and find that novel, dynamic biomarkers exist that are predictive of systemic disease control and overall survival.