TEmoX: Classification of Textual Emotion Using Ensemble of Transformers
Avishek Das, Mohammed Moshiul Hoque, Omar Sharif, M. Ali Akber Dewan, Nazmul Siddique
Abstract
Textual emotion classification (TxtEC) refers to the classification of emotion expressed by individuals in textual form. The widespread use of the Internet and numerous Web 2.0 applications has emerged in an expeditious growth of textual interactions. However, determining emotion from texts is challenging due to their unorganized, unstructured, and disordered forms. While research in textual emotion classification has made considerable breakthroughs for high-resource languages, it is yet challenging for low-resource languages like Bengali. This work presents a transformer-based ensemble approach (called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TEmoX</i> ) to categorize Bengali textual data into six integral emotions: joy, anger, disgust, fear, sadness, and surprise. This research investigates 38 classifier models developed using four machine learning LR, RF, MNB, SVM, three deep-learning CNN, BiLSTM, CNN+BiLSTM, five transformer-based m-BERT, XLM-R, Bangla-BERT-1, Bangla-BERT-2, and Indic-DistilBERT techniques with two ensemble strategies and three embedding techniques. The developed models are trained, tuned, and tested on the three versions of the Bengali emotion text corpus BEmoC-v1, BEmoC-v2, BEmoC-v3. The experimental outcomes reveal that the weighted ensemble of four transformer models En-22: Bangla-BERT-2, XLM-R, Indic-DistilBERT, Bangla-BERT-1 outperforms the baseline models and existing methods by providing the maximum weighted <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i> 1-score (80.24%) on BEmoC-v3. The dataset, models, and fractions of codes are available at https://github.com/avishek-018/TEmoX.