Deep Reinforcement Learning for Stabilization of Large-Scale Probabilistic Boolean Networks
Sotiris Moschoyiannis, Evangelos Chatzaroulas, Vytenis Šliogeris, Yuhu Wu
Abstract
The ability to direct a probabilistic Boolean network (PBN) to the desired state is important to applications such as targeted therapeutics in cancer biology. Reinforcement learning (RL) has been proposed as a framework that solves a discrete-time optimal control problem cast as a Markov decision process. We focus on an integrative framework powered by a model-free deep RL method that can address different flavors of the control problem (e.g., with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">or</i> without control inputs; having attractor states <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">or</i> a subset of the state space as the target domain). The method is agnostic to the distribution of probabilities for the next state; hence, it does not use the probability transition matrix. The time complexity is only <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">linear</i> on the time steps or interactions between the agent (deep RL) and the environment (PBN), during training. Indeed, we explore the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">scalability</i> of the deep RL approach to (set) stabilization of large-scale PBNs and demonstrate successful control on large networks, including a metastatic melanoma PBN with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">200 nodes</i> .