LLM-Driven Testing for Autonomous Driving Scenarios
Nenad Petrović, Krzysztof Lebioda, Vahid Zolfaghari, André Schamschurko, Sven Kirchner, Nils Purschke, Fengjunjie Pan, Alois Knoll
Abstract
In this paper, we explore the potential of leveraging Large Language Models (LLMs) for automated test generation based on free-form textual descriptions in area of automotive. As outcome, we implement a prototype and evaluate the proposed approach on autonomous driving feature scenarios in CARLA open-source simulation environment. Two pre-trained LLMs are taken into account for comparative evaluation: GPT-4 and Llama3. According to the achieved results, GPT-4 outperforms Llama3, while the presented approach speeds-up the process of testing (more than 10 times) and reduces cognitive load thanks to automated code generation and adoption of flexible simulation environment for quick evaluation.