SVCE: Shapley Value Guided Counterfactual Explanation for Machine Learning-Based Autonomous Driving
Meng Li, Hengyang Sun, Yanjun Huang, Hong Chen
Abstract
The explainability of complex machine-learning models is becoming increasingly significant in safety-critical domains such as autonomous driving. In this context, counterfactual explanation (CE), as an effective explainability method in explainable artificial intelligence, plays an important role. It aims to identify minimal alterations to input that can change the model’s output, thereby revealing key factors influencing model decisions. However, generating counterfactual samples might involve manually selecting input features, potentially leading to suboptimal and biased explanations. This study introduces a feature contribution guided CE generation framework to address this issue. Our method utilizes feature contributions based on Shapley values to guide the model’s focus on the most influential features. This enables end-users to quickly pinpoint the search direction in generating CEs (e.g., prioritizing the most critical features) and producing representative CEs. To comprehensively evaluate our method, we conducted experimental validation on two representative machine learning models: autonomous driving decision-making using Deep Q-Network and lane-changing prediction using deep learning. In addition, we conducted a user-centered study to evaluate the practical applicability of the SVCE in autonomous driving scenarios, which serves as a crucial validation of the presented SVCE. The results show that SVCE can help users understand and diagnose the model.