Open Spatio-Temporal Foundation Models for Traffic Prediction

Abstract

Accurate traffic forecasting is crucial for effective urban planning and transportation management, enabling efficient resource allocation and enhanced travel experiences. However, existing models often face limitations in generalization, struggling with zero-shot prediction on unseen regions and cities, as well as diminished long-term accuracy. This is primarily due to the inherent challenges in handling the spatial and temporal heterogeneity of traffic data, coupled with the significant distribution shift across time and space. In this work, we aim to unlock new possibilities for building versatile, resilient and adaptive spatio-temporal foundation models for traffic prediction. We introduce OpenCity, a foundation model that captures underlying spatio-temporal patterns from diverse data, facilitating zero-shot generalization across urban environments. OpenCity integrates Transformers with graph neural networks to capture complex spatio-temporal dependencies in traffic data. By pre-training OpenCity on large-scale, heterogeneous traffic data from web platforms, we enable the model to learn rich, generalizable representations that can be seamlessly applied to a wide range of traffic forecasting scenarios. Experiments show OpenCity excels in zero-shot prediction and exhibits scaling laws, highlighting its potential as a universal one-for-all traffic prediction solution adaptable to new urban contexts with minimal overhead. Source codes are available at: https://github.com/HKUDS/OpenCity

Topics & Concepts