Model Explanations with Differential Privacy
Neel Patel, Reza Shokri, Yair Zick
Abstract
Using machine learning models in critical decision-making processes has given rise to a call for algorithmic transparency. Model explanations, however, might leak information about the sensitive data used to train and explain the model, undermining data privacy. We focus on black-box feature-based model explanations, which locally approximate the model around the point of interest, using potentially sensitive data. We design differentially private local approximation mechanisms, and evaluate their effect on explanation quality. To protect training data, we use existing differentially private learning algorithms. However, to protect the privacy of data which is used during the local approximation, we design an adaptive differentially private algorithm, which finds the minimal privacy budget required to produce accurate explanations. Both empirically and analytically, we evaluate the impact of the randomness needed in differential privacy algorithms on the fidelity of model explanations.