Exploring BERT and ELMo for Bangla Spam SMS Dataset Creation and Detection

Abstract

The proliferation of digital communication and social media platforms has led to an upsurge in spam messages across various languages, including Bangla. These intrusive messages not only cause user annoyance but also elevate the risk of important messages going unnoticed. To address this concern, we harnessed the power of natural language processing (NLP) techniques in conjunction with machine-learning models. This study delves into the efficacy of diverse machine-learning and deep-learning models in detecting spam messages in the Bangla language. Particularly, deep-learning methodologies such as ELMo and BERT have exhibited promising results in spam detection across other languages. However, the realm of Bangla spam detection remains underexplored, with a dearth of deep-learning approaches. To bridge this gap, we constructed a unique Bangla SMS dataset, given the scarcity of available Bangla Spam SMS datasets, and proceeded to evaluate the performance of ELMo, BERT, and conventional machine learning models. The findings from our experiments underscore the superior performance of deep learning-based models over traditional methods, with Elmo surpassing BERT in Bangla spam detection, achieving an impressive accuracy of approximately 94%. This endeavor bears the potential to not only enhance SMS utilization efficiency but also bolster cybersecurity measures against phishing attacks.

Topics & Concepts