Model of Gradient Boosting Random Forest Prediction
Zhidong Zhang, Xiubin Zhu, Ding Liu
Abstract
Random forests (RF) is an ensemble classification approach, which is easy to use and is helpful to avoid over-fitting. However, in the complex data environment, its prediction accuracy could be deteriorated. Gradient boosting decision tree (GBDT) is another widely used in classification problems because of its high prediction accuracy and interpretability. In order to improve the performance of random forest in solving classification problems, this paper proposes a gradient boosting random forest (GBRF) algorithm. GBRF algorithm employs the idea of gradient to optimize decision tree at the bottom of random forest into gradient boosting decision tree, which improves the prediction accuracy of the bottom tree, and thus improves the prediction performance of random forest. To verify the effectiveness of GBRF algorithm, data sets in UCI and KEEL are used for group testing. The results show that the classification accuracy of GBRF algorithm has a higher prediction accuracy improvement compared to random forest and the performance improvement is more than 5 percent, which indicates that GBRF algorithm performs better than the original random forest.