Explaining and visualizing black-box models through counterfactual paths
Bastian Pfeifer, Mateusz Krzyziński, Hubert Baniecki, Andreas Holzinger, Przemysław Biecek
Abstract
Abstract Explainable AI (XAI) is an increasingly important area of machine learning research, which aims to make black-box models transparent and interpretable. In this paper, we propose a novel approach to XAI that uses the so-called counterfactual paths for model-agnostic global explanations. The algorithm measures feature importance by identifying sequential permutations of features that most influence changes in model predictions. It is particularly suitable for generating explanations based on counterfactual paths in knowledge graphs incorporating domain knowledge. Counterfactual paths introduce an additional graph dimension to current XAI methods in both explaining and visualizing black-box models. Experiments with synthetic and bio-medical data demonstrate the practical applicability of our approach.