A Comparison of BiLSTM, BERT, and Ensemble Method for Emotion Recognition on Indonesian Product Reviews

Abstract

Emotion recognition within online product reviews is pivotal for enhancing business strategies and customer insights. Recognizing emotions in Indonesian, a language rich in nuances and expressions presents significant challenges due to its complex linguistic structure and the scarcity of tailored datasets. This study aims to bridge this gap by evaluating the effectiveness of Bidirectional Long Short-Term Memory (BiLSTM), Bidirectional Encoder Representations from Transformers (BERT), and ensemble methods in analyzing emotions from Indonesian product reviews using the PRDECT-ID dataset. Extensive fine-tuning across 23 BERT con- figurations and multiple BiLSTM preprocessing combinations was conducted to adapt these models to the Indonesian linguistic context. Each model was assessed based on the F1 score. The BiLSTM model was particularly effective in configurations with complex preprocessing, achieving an optimal F1 score of 61% through advanced noise removal, stemming, and a modified stop- words list. Conversely, the minimally preprocessed fine-tuned ’base-p2’ and ’large-p1’ BERT variants achieved F1 scores of 72% and 73%, respectively, both surpassing the previous best result of 71%. This research also explored 4 ensemble methods, combin- ing the strengths of the best-performing BiLSTM and BERT models using both soft-voting and stacked generalization techniques. The unweighted stacked generalization achieved a 74% F1 score, while the weighted method excelled with the highest F1 score of 75%, surpassing other models and highlighting the advantages of strategic model integration. This research significantly advances the development of NLP models for Indonesian text, demonstrating how tailored deep-learning approaches can effectively enhance emotion recognition accuracy.

Topics & Concepts