Classifying Sentiments on Social Media Texts: A GPT-4 Preliminary Study
Lany L. Maceda, Jennifer L. Llovido, Miles Biago Artiaga, Mideth Abisado
Abstract
In today's digital age, social media has become a hub for people to express their thoughts and feelings. Sentiment classification discerns public opinions and trends to understand their sentiments towards a certain topic. Often, achieving accurate sentiment classifications in large datasets necessitate the use of human-annotated training data which can be costly and time-consuming. Large Language Models (LLMs) like the Generative Pre-trained models by OpenAI have surged in popularity due to its capabilities in understanding the given tasks. In this preliminary study, we report the performance of the latest OpenAI GPT-4 using zero- and one-shot learning approaches on classifying sentiments when fed with social media dataset. Notably, the latter approach written in English which mimics the instructions designed for human annotators, achieved a substantial agreement (k = 0.77) with human annotations, displaying high accuracy, precision, and recall accordingly even without explicit training data. Meanwhile, the fine-tuned mBERT resulted to lower evaluation scores than the GPT-4. Our findings provide foundational insights into the strengths and limitations of GPT-4 for sentiment classification in a social media dataset, setting the groundwork for broad future research in this field.