Disambiguating Clinical Abbreviations using Pre-trained Word Embeddings
Areej Jaber, Paloma Martı́nez
Abstract
Abbreviations are extensively used in electronic health records (EHR) of patients as \nwell as medical documentation, reaching 30-50% of the words in clinical narrative. There \nare more than 197,000 unique medical abbreviations found in the clinical text and their \nmeanings vary depending on the context in which they are used. Since data in electronic \nhealth records could be shareable across health information systems (hospitals, primary care centers, etc.) as well as others such as insurance companies information systems, it is essential determining the correct meaning of the abbreviations to avoid misunderstandings. \nClinical abbreviations have specific characteristic that do not follow any standard \nrules for creating them. This makes it complicated to find said abbreviations and corresponding meanings. Furthermore, there is an added difficulty to working with clinical data due to privacy reasons, since it is essential to have them in order to develop and test algorithms. \nWord sense disambiguation (WSD) is an essential task in natural language processing \n(NLP) applications such as information extraction, chatbots and summarization systems \namong others. WSD aims to identify the correct meaning of the ambiguous word which \nhas more than one meaning. Disambiguating clinical abbreviations is a type of lexical \nsample WSD task. Previous research works adopted supervised, unsupervised and \nKnowledge-based (KB) approaches to disambiguate clinical abbreviations. This thesis \naims to propose a classification model that apart from disambiguating well known abbreviations also disambiguates rare and unseen abbreviations using the most recent deep neural network architectures for language modeling. \nIn clinical abbreviation disambiguation several resources and disambiguation models \nwere encountered. Different classification approaches used to disambiguate the clinical \nabbreviations were investigated in this thesis. Considering that computers do not directly \nunderstand texts, different data representations were implemented to capture the meaning of the words. Since it is also necessary to measure the performance of algorithms, the evaluation measurements used are discussed. \nAs the different solutions proposed to clinical WSD we have explored static word embeddings data representation on 13 English clinical abbreviations of the UMN data set (from University of Minnesota) by testing traditional supervised machine learning algorithms separately for each abbreviation. Moreover, we have utilized a transformer-base pretrained model that was fine-tuned as a multi-classification classifier for the whole data set (75 abbreviations of the UMN data set). The aim of implementing just one multi-class classifier is to predict rare and unseen abbreviations that are most common in clinical \nnarrative. Additionally, other experiments were conducted for a different type of abbreviations \n(scientific abbreviations and acronyms) by defining a hybrid approach composed \nof supervised and knowledge-based approaches. \nMost previous works tend to build a separated classifier for each clinical abbreviation, \ntending to leverage different data resources to overcome the data acquisition bottleneck. \nHowever, those models were restricted to disambiguate terms that have been \nseen in trained data. Meanwhile, based on our results, transfer learning by fine-tuning a \ntransformer-based model could predict rare and unseen abbreviations. A remaining challenge for future work is to improve the model to automate the disambiguation of clinical abbreviations on run-time systems by implementing self-supervised learning models.