Learning Driven Decision Intelligence for Autonomous Driving Through Multimodal Understanding World Modeling and Policy Optimization
Fengyuan Zhang, Jie-Si Yang
Abstract
Autonomous driving represents one of the most challenging applications of artificial intelligence (AI), requiring sophisticated decision-making capabilities that integrate perception, prediction, and planning under dynamic and uncertain conditions. Recent advances in learning-driven approaches have demonstrated remarkable potential in addressing these challenges through multimodal understanding, world modeling, and policy optimization. Deep learning (DL) techniques enable vehicles to process heterogeneous sensory inputs including camera images, LiDAR point clouds, and radar signals to construct comprehensive environmental representations. World models provide predictive frameworks that simulate future scenarios and potential outcomes, allowing autonomous systems to anticipate complex traffic dynamics and make informed decisions. Reinforcement learning (RL) and imitation learning methods optimize driving policies through interaction with real and simulated environments, progressively improving decision quality and safety. This review examines the current state of learning-driven decision intelligence in autonomous driving, analyzing how multimodal perception architectures extract meaningful features from diverse sensor modalities, how world modeling techniques enable forward-looking planning capabilities, and how policy optimization frameworks translate environmental understanding into safe and efficient driving behaviors. We synthesize recent developments in transformer-based architectures, neural rendering approaches, and end-to-end learning systems that directly map sensory inputs to control actions. The integration of these components presents both significant opportunities and substantial challenges, including handling distribution shifts between training and deployment scenarios, ensuring robustness to adversarial conditions, and achieving the safety guarantees required for real-world deployment.