Leveraging LightGBM for Categorical Big Data

Abstract

LightGBM is a popular Gradient Boosted Decision Tree implementation for classification and regression tasks. Our contribution is to answer a research question regarding LightGBM. We would like to know which alternative yields better performance for classifying highly imbalanced Big Data with high-cardinality categorical features: relying entirely on LightGBM’s Exclusive Feature Bundling as a way to encode categorical features, or using LightGBM’s built-in encoding for categorical features? Our study of LightGBM revealed two alternatives for a Big Data classification task to do anomaly detection. We may one-hot encode the data into a sparse representation, and then rely entirely on LightGBM’s Exclusive Feature Bundling to complete encoding of the categorical features. Exclusive Feature Bundling is LightGBM’s optimization technique that exploits sparsity in features for reducing the dimensionality of a dataset. On the other hand, also because our data has categorical features, it is a candidate for LightGBM’s built-in encoding technique for categorical features. Since we did not find a clear indication in a survey of related work for which direction to take – using Exclusive Feature Bundling or using LightGBM’s built-in encoding for categorical features – we experiment with these options to determine the best one for highly imbalanced Big Data. Furthermore, we show LightGBM’s built-in encoding is best in a statistically significant sense. Our work is important because it fills a gap in LightGBM-related literature on how to best handle categorical features in imbalanced Big Data with high-cardinality categorical features.

Topics & Concepts