Transformer-based Multi-Task Learning for Adverse Effect Mention Analysis in Tweets
George-Andrei Dima, Dumitru-Clementin Cercel, Mihai Dascălu
Abstract
While social media gains a broader traction, valuable insights and opinions on various topics representative for a wider audience can be automatically extracted using state-of-theart Natural Language Processing techniques. Of particular interest in the healthcare domain are adverse drug effects, which may be introduced in online posts, and can be effectively centralized and investigated. This paper presents our Multi-Task Learning architecture using pretrained Transformer-based language models employed for the Social Media Mining for Health Applications Shared Task 2021, where we tackle the three subtasks of Task 1, namely: classification of tweets containing adverse effects (subtask 1a), extraction of text spans containing adverse effects (subtask 1b), and adverse effects resolution (subtask 1c). Our best performing model ranked first on the test set at subtask 1b with an F 1score of 51% (P = 51%; R = 51%). Promising results were obtained on subtask 1a (F 1 -score = 44%; P = 45%; R = 43%), whereas subtask 1c was by far the most difficult task and an F 1score of only 17% (P = 17%; R = 18%) was obtained.