Large language model-driven probabilistic trajectory prediction in the Internet of Things using spatio-temporal encoding and normalizing flows

Abstract

The convergence of the Internet of Things (IoT) and foundation models, particularly Large Language Models (LLMs), heralds a new era of intelligent and adaptive systems. This investigation explores the integration of LLMs and IoT for probabilistic vehicle trajectory prediction, a critical component in autonomous driving. Traditional deterministic models often fail to capture the complexity and uncertainty of real-world driving environments. To address these limitations, we propose an innovative approach that leverages LLM-driven spatio-temporal encoding and normalizing flows. The proposed method extracts latent motion patterns from historical trajectory data using LLMs, providing a nuanced understanding of vehicle dynamics. Advanced spatio-temporal encoders are employed for comprehensive feature extraction, capturing temporal dependencies and spatial relationships. Additionally, scaled dot-product attention is utilized for effective multimodal feature fusion, enhancing the integration of diverse data sources. The normalizing flows framework enhances the model's ability to capture complex, multimodal probability distributions of future trajectories by constructing an invertible transformation network. The network systematically maps simple base distributions to intricate target distributions through reversible transformations, significantly improving prediction accuracy. Evaluation of the nuScenes dataset demonstrates substantial improvements over existing methods. The results underscore the model's capability to generate diverse and realistic trajectory distributions, paving the way for safer and more efficient autonomous driving systems.

Topics & Concepts