MSGFormer: Revolutionizing Traffic Flow Prediction With Multiscale and Gated Transformer Architecture

Abstract

Traffic flow prediction has emerged as a critical component in advancing smart cities. Nevertheless, precisely forecasting traffic flow remains a formidable challenge, attributable to the intricate and dynamic spatiotemporal interdependencies inherent within traffic data. Contemporary methodologies often overlook the different impacts exerted at each timestamp and treat temporal correlations, rendering them incapable of extracting temporal patterns at multiple scales. Furthermore, these approaches fail to account for the neighboring and functional relationships among nodes within the spatial module. In this work, we introduce an innovative Multiscale and Gated transFormer (MSGFormer) architecture, MSGFormer, to overcome the inherent limitations for accurate traffic flow estimation. We have introduced a multiscale sampling strategy wherein we sample from the original data at three scales: 1) recent; 2) daily; and 3) weekly. This approach enables the generation of corresponding subsequences that encapsulate temporal information across different granularities. Subsequently, each generated subsequence is projected into a latent space and systematically combined with positional, temporal, and spatial embeddings. The positional embedding comprises relative positional embedding and time stamp embedding corresponding to days and weeks, aiming at capturing the sequential and cyclical characteristics of the data. Furthermore, at the inception of the transformer encoder, a gated unit, composed of a neighboring mask and a functioning mask, is employed to capture both static and dynamic spatial correlations effectively. Comprehensive experiments have been conducted on four real-world traffic data sets. The experimental results robustly validate that our model attains significantly higher predictive accuracy in comparison to other baseline models.

Topics & Concepts