Spatio-Temporal Capsule-Based Reinforcement Learning for Mobility-on-Demand Coordination
Suining He, Kang G. Shin
Abstract
As an alternative means of convenient and smart transportation, mobility-on-demand (MOD), typified by online ride-sharing and connected taxicabs, has been rapidly growing and spreading worldwide. The large volume of complex traffic and the uncertainty of market supplies/demands have made it essential for many MOD service providers to <i>proactively</i> dispatch vehicles towards ride-seekers. To meet this need effectively, we propose <monospace>STRide</monospace> , an MOD coordination learning mechanism reinforced spatio-temporally with capsules. We formalize the adaptive coordination of vehicles into a reinforcement learning framework. <monospace>STRide</monospace> incorporates spatial and temporal distributions of supplies (vehicles) and demands (ride requests), customers’ preferences and other external factors. A novel spatio-temporal capsule neural network is designed to predict the provider’s rewards based on MOD network states, vehicles and their dispatch actions. This way, the MOD platform adapts itself to the supply-demand dynamics with the best potential rewards. We have conducted extensive data analytics and experimental evaluation with five large-scale datasets ( <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 27 million rides from Uber, NYC/Chicago Taxis, Didi and Car2Go). <monospace>STRide</monospace> is shown to outperform state-of-the-arts, substantially reducing request-rejection rate and passenger waiting time, and also increasing the service provider’s profits.