Autonomous Vehicle Evaluation: A Comprehensive Survey on Modeling and Simulation Approaches
Hesham Alghodhaifi, Sridhar Lakshmanan
Abstract
In recent years, autonomous vehicles (AVs), which observe the driving environment and lead a few or all of the driving duties, have garnered tremendous success. The field of AVs has been developing rapidly and has found many applications. As a safety requirement by policymakers, these vehicles must be evaluated before their deployment. The evaluation process for AVs is challenging because crashes are rare events, and AVs can escape passing predefined test scenarios. Therefore, capturing crashes and creating real test scenarios should be considered in order to have an evaluation approach that represents the real-world scenarios. One evaluation approach is based on the naturalistic field operational test (N-FOT), in which prototype AVs are driven by volunteers or test engineers on the roads. Unfortunately, this approach is time-consuming and costly because one needs to drive thousands of miles to experience a police-reported collision and nearly millions of miles for a fatal crash. Another approach is the Accelerated Evaluation method. The core idea of the Accelerated Evaluation approach is to modify the statistics of naturalistic driving so that safety-critical events are emphasized. This paper presents a brief survey of the advances that have occurred in the area of the evaluation of partly or fully AVs, starting with naturalistic field operational tests (N-FOTs). The review goes on to cover test matrix evaluation, worst-case scenario evaluation (WCSE), Monte Carlo simulations, and accelerated evaluation (AE). We also present all the simulation-based and agent-based modeling approaches that do not follow any evaluation protocol listed above. This study provides a scientific analysis of each of the evaluation techniques, focusing on their advantages/disadvantages, inherent restrictions, practicability, and optimality. The results reveal that the accelerated evaluation approach outperforms naturalistic field operational tests (N-FOTs), test matrix evaluation, worst-case scenario evaluation (WCSE), Monte Carlo simulations methods in some of the car-following, and lane-change studies when using specific models. Moreover, the agent-based model and augmented and virtual reality approaches show promising results in AVs evaluation. Furthermore, integrating machine and deep learning into the available AV evaluation methods can improve its performance and generate encouraging outcomes.