Developing a New Classifier for Automated Identification of Incivility in Social Media
Sam Davidson, Qiusi Sun, Magdalena Wojcieszak
Abstract
Incivility is not only prevalent on online social media platforms, but also has concrete effects on individual users, online groups, the platforms themselves, and the society at large. Given the prevalence and effects of online incivility, and the challenges involved in humanbased incivility detection, it is urgent to develop validated and versatile automatic approaches to identifying uncivil posts and comments. This project advances both a neural, BERT-based classifier as well as a logistic regression classifier to identify uncivil comments. The classifier is trained on a dataset of Reddit posts, which are annotated for incivility, and further expanded using a combination of labeled data from Reddit and Twitter. Our best performing model achieves an F 1 of 0.802 on our Reddit test set. The final model is not only applicable across social media platforms and their distinct data structures, but also computationally versatile, and -as such -ready to be used on vast volumes of online data. All trained models and annotated data are made available to the research community.