Autograding of Programming Skills
Nakka Narmada, Peeta Basa Pati
Abstract
To evaluate a learner’s knowledge of programming language skills, assessments are given. Grading of these is usually done manually which not only is tedious but prone to error due to repetition and fatigue. In this work, we employ pre-trained language models to perform automated grading of "C" programming language. Embeddings from different transformers on pre-assessed codes are used as feature vectors to train a wide range of regressors for the scoring task. Root-mean-square error (RMSE) is the metric utilized to compare the scores of these regressors. It’s observed that embeddings from T5-model with CatBoost regressor gives the least error around 15%.
Topics & Concepts
Computer scienceGrading (engineering)Mean squared errorTransformerFeature engineeringArtificial intelligenceNatural language processingMachine learningSpeech recognitionStatisticsDeep learningMathematicsVoltageCivil engineeringPhysicsQuantum mechanicsEngineeringSoftware Engineering ResearchSoftware Testing and Debugging TechniquesSoftware Reliability and Analysis Research