A Subgraph-Based Hierarchical Q-Learning Approach to Optimal Resource Scheduling for Complex Industrial Networks
Kexin Zhang, Qing Gao, Jinhu Lü, Maciej Ogorzałek, Yue Deng
Abstract
This paper proposes a subgraph-based hierarchical Q-learning network (SgHQN) approach to solve the optimal resource scheduling problem for complex industrial networks. In the industrial network, each connection between two individual stations has limited communication bandwidth, while each station has limited computing and storage capability and is only accessible to its local information. The resource packages flowing within the industrial network are treated as agents that have different sizes and different levels of decision-making priority. This makes the industrial resource scheduling problem on the industrial network a multi-level decision-making problem with information asymmetry. Specifically, the resource packages with lower decision-making priority have knowledge of the decisions made by those with higher priority, but not vice versa. To solve this resource scheduling problem with information asymmetry, an SgHQN model is developed by exploiting partial observations. It is found that the proposed SgHQN can be used to solve resource scheduling problems for general industrial networks. Numerical experiments simulating industrial scheduling scenarios demonstrate the effectiveness and advantages of our method.