Unbiased Measurement of Feature Importance in Tree-Based Methods
Zhengze Zhou, Giles Hooker
Abstract
We propose a modification that corrects for split-improvement variable importance measures in Random Forests and other tree-based methods. These methods have been shown to be biased towards increasing the importance of features with more potential splits. We show that by appropriately incorporating split-improvement as measured on out of sample data, this bias can be corrected yielding better summaries and screening tools.
Topics & Concepts
Feature (linguistics)Tree (set theory)Random forestComputer scienceVariable (mathematics)Data miningPattern recognition (psychology)StatisticsMathematicsArtificial intelligenceMachine learningPhilosophyMathematical analysisLinguisticsStatistical Methods and InferenceBayesian Modeling and Causal InferenceStatistical Methods and Bayesian Inference