An Empirical Evaluation of Stacked Ensembles With Different Meta-Learners in Imbalanced Classification

Abstract

The selection of a meta-learner determines the success of a stacked ensemble as the meta-learner is responsible for the final predictions of the stacked ensemble. Unfortunately, in imbalanced classification, selecting an appropriate and well-performing meta-learner of stacked ensemble is not straightforward as different meta-learners are advocated by different researchers. To investigate and identify a well-performing type of meta-learner in stacked ensemble for imbalanced classification, an experiment consisting of 19 meta-learners was conducted, detailed in this paper. Among the 19 meta-learners of stacked ensembles, a new weighted combination-based meta-learner that maximizes the H-measure during the training of stacked ensemble was first introduced and implemented in the empirical evaluation of this paper. The classification performances of stacked ensembles with 19 different meta-learners were recorded using both the area under the receiver operating characteristic curve (AUC) and H-measure (a metric that overcomes the deficiencies of the AUC). The weighted combination-based meta-learners of stacked ensembles have better classification performances on imbalanced datasets when compared to bagging-based, boosting-based, Decision Trees, Support Vector Machines, Naive Bayes, and Feedforward Neural Network meta-learners. Thus, the adoption of weighted combination-based meta-learners in stacked ensembles is recommended for their better performance on imbalanced datasets. Also, based on the empirical results, we identified better-performing meta-learners (such as the AUC maximizing meta-learner and the H-measure maximizing meta-learner) than the widely adopted meta-learner - Logistic Regression - in imbalanced classification.

Topics & Concepts