Multiscale mathematical model-informed reinforcement learning optimizes combination treatment scheduling in glioblastoma evolution
Zeming Liu, Ji Zhang, Liu Hong, Qing Nie, Xiaoqiang Sun
Abstract
Dynamic tumor-microenvironment interactions greatly affect growth and drug resistance, highlighting the importance and challenge of developing mathematical models to optimize treatment schedules. Here, we describe a multiscale mathematical model-informed reinforcement learning (M4RL) framework to simulate dynamic tumor-microenvironment interactions and optimize drug combination scheduling. We first develop a multiscale agent-based model (MSABM) for a critical biological scenario where interactions between tumor-associated macrophages (TAMs) and tumor cells (TCs) underlie immunotherapy resistance in glioblastoma. Next, we learn a surrogate model based on Fokker-Planck equations for the MSABM using a physics-informed neural network approach. We then design a surrogate model-based reinforcement learning method, using an efficient parallel actor-critic algorithm, to predict optimal scheduling of combination treatments. The most effective regimen of dynamic combination of CSF1R inhibitor (targeting TAMs) and IGF1R inhibitor (targeting TCs) is identified and then verified using spatial transcriptomic data. Overall, the M4RL framework introduces a computational approach for characterizing tumor-microenvironment interactions and optimizing dynamic scheduling of drug combinations.