Feature Selection and Classification – A Probabilistic Wrapper Approach
Huan Liu, Rudy Setiono
Abstract
Feature selection is defined as a problem to find a minimum set of M features for an inductive algorithm to achieve the highest predictive accuracy from the data described by the original N features where M ≤ N. A probabilistic wrapper model is proposed as another method besides the exhaustive search and the heuristic approach. The aim of this model is to avoid local minima and exhaustive search. The highest predictive accuracy is the criterion in search of the smallest M. Analysis and experiments show that this model can effectively find relevant features and remove irrelevant ones in the context of improving the predictive accuracy of an induction algorithm. It is simple, straightforward, and providing fast solutions while searching for the optimal. The applications of such a model, its future work and some related issues are also discussed.