A Reinforcement Learning Method for Multiasset Roadway Improvement Scheduling Considering Traffic Impacts
Weiwen Zhou, Elise Miller-Hooks, Kostas G. Papakonstantinou, Shelley M. Stoffels, Sue McNeil
Abstract
Maintaining roadway pavements and bridge decks is key to providing high levels of service for road users. However, improvement actions incur downtime. These actions are typically scheduled by asset class, yet implemented on any asset type, they have network-wide impacts on traffic performance. This paper presents a bilevel program wherein the upper level involves a Markov decision process (MDP) through which potential roadway improvement actions across asset classes are prioritized and scheduled. The MDP approach considers uncertainty in component deterioration effects, while incorporating the benefits of implemented improvement actions. The upper level takes as input traffic flow estimates obtained from a lower-level user equilibrium traffic formulation that recognizes changes in capacities determined by decisions taken in the upper level. Because an exact solution of this bilevel, stochastic, dynamic program is formidable, a deep reinforcement learning (DRL) method is developed. The model and solution methodology were tested on a hypothetical problem from the literature. The importance of obtaining optimal activity plans that account for downtime effects, traffic congestion impacts, uncertainty in deterioration processes, and multiasset classes is demonstrated.