Causally-Informed Instance-Wise Feature Selection for Explaining Visual Classifiers
Li Kuo Tan
Abstract
We propose a novel interpretability framework that integrates instance-wise feature selection with causal reasoning to explain decisions made by black-box image classifiers. Instead of relying on feature importance or mutual information, our method identifies input regions that exert the greatest causal influence on model predictions. Causal influence is formalized using a structural causal model and quantified via a conditional mutual information term. To optimize this objective efficiently, we employ continuous subset sampling and the matrix-based Rényi's α-order entropy functional. The resulting explanations are compact, semantically meaningful, and causally grounded. Experiments across multiple vision datasets demonstrate that our method outperforms existing baselines in terms of predictive fidelity.