Enhancing automated essay scoring by leveraging LSTM networks with hyper-parameter tuned word embeddings and fine-tuned LLMs

Abstract

Abstract Automated Essay Scoring (AES) as a classification task is an effective way to categorize essays based on predefined rubrics or score ranges. It focuses on predicting a class label (score or grade) rather than assigning a precise numerical score, offering advantages in simplicity, adaptability, and consistency. Traditionally, LSTM with word embeddings that are trained on large general corpora have been used for the classification task. Domain-tailored word embeddings provide a significant added advantage to capture the specific terminology and semantic relationships relevant to a particular field, domain, or application. Hence, in this study we proposed two approaches: (1) hyper-parameter-tuned Word2Vec embeddings catered to the need of AES system are fed into Long Short-Term Memory (LSTM) network, and (2) fine-tuning a Large Language Model from Meta AI (Llama) on the Automated Student Assessment Prize (ASAP) dataset. The results indicate that hyper parameter tuning helped to generate domain tailored embeddings which significantly improves model accuracy by reducing inconsistencies in essay scoring. Additionally, fine-tuning the Large Language Model’s (LLMs) further improves system performance. These results show that the presented approaches significantly helped to outperform the limitations of the methods used in existing literature, leading to higher accuracy in AES system.

Topics & Concepts