Significant Sampling for Shortest Path Routing: A Deep Reinforcement Learning Solution
Yulin Shao, Arman Rezaee, Soung Chang Liew, Vincent W. S. Chan
Abstract
Significant sampling is an adaptive monitoring technique proposed for highly dynamic networks with centralized network management and control systems. The essential spirit of significant sampling is to collect and disseminate network state information when it is of significant value to the optimal operation of the network, and in particular when it helps identify the shortest routes. Discovering the optimal sampling policy that specifies the optimal sampling frequency is referred to as the significant sampling problem. Modeling the problem as a Markov Decision process, this paper puts forth a deep reinforcement learning (DRL) approach to tackle the significant sampling problem. This approach is more flexible and general than prior approaches as it can accommodate a diverse set of network environments. Experimental results show that, 1) by following the objectives set in the prior work, our DRL approach can achieve performance comparable to their analytically derived policy φ' - unlike the prior approach, our approach is model-free and unaware of the underlying traffic model; 2) by appropriately modifying the objective functions, we obtain a new policy which addresses the never-sample problem of policy φ', consequently reducing the overall cost; 3) our DRL approach works well under different stochastic variations of the network environment - it can provide good solutions under complex network environments where analytically tractable solutions are not feasible.