Systematic Literature Review of Stemming and Lemmatization Performance for Sentence Similarity
Rio Pramana, Débora, Jonathan Jansen Subroto, Alexander Agung Santoso Gunawan, Anderies
Abstract
In today’s era, where the Internet is a huge part of people’s life, Information Retrieval (IR) is as important as ever for people to retrieve relevant information in a quick way. The sentence similarity task is an integral aspect of IR. Improving the performance of this task will also improve IR. To help achieve this, stemming and lemmatization was made. However, it is still unclear to many which one is the best to use for sentence similarity tasks. Therefore, the purpose of this study is to find which preprocessing technique (stemming and lemmatization) is best for sentence similarity tasks. In this study, the authors would like to conduct a Systematic Literature Review (SLR) on stemming and lemmatization based on many previous studies related to this topic. Previous studies have tried to assess and compare both preprocessing techniques using many evaluation methods, and it is found that a lot of factors go into deciding which preprocessing technique (stemming or lemmatization) is the best option. In general, the authors conclude lemmatization is considered the best option for sentence similarity tasks since it produces better results than stemming. However, if speed optimization is imperative, then stemming is the better option since its computational speed is faster.