A Comprehensive Comparative Analysis of Deep Learning Models for Student Performance Prediction in Virtual Learning Environments: Leveraging the OULA Dataset and Advanced Resampling Techniques

Abstract

Predicting student performance in Virtual Learning Environments (VLEs) has become increasingly important with the growth of online education. Early identification of at-risk students enables timely interventions to improve academic outcomes. This study evaluates the performance of several Deep Learning (DL) models for tabular data, including ResNet, NODE, AutoInt, TabNet, TabTransformer (TT), SAINT, and GatedTabTransformer (GTT). Moreover, it examines the role of resampling techniques, including SMOTE, ROS, ADASYN, RUS, and Tomek-Links, in addressing class imbalance. Using the OULA dataset, eight experiments were conducted for binary and multi-class classification tasks, testing different feature combinations: (1) behavioral; (2) demographic and behavioral; (3) academic and behavioral; and (4) demographic, academic, and behavioral. The results indicate that the incorporation of a comprehensive set of characteristics significantly enhances the performance of the model, with academic characteristics proving more predictive than demographic characteristics. The SAINT model achieved the highest performance in binary classification (94.33% accuracy), leveraging its ability to capture simple yet meaningful feature interactions. For multi-class classification, SAINT again outperformed other models, achieving an accuracy of 73.22% when using Tomek-Links method, excelling in managing complex feature interactions and underrepresented classes such as “Distinction”. Statistical analysis using the Friedman aligned ranks test and the Nemenyi post-hoc test was conducted to compare model performance based on F1-scores from multiple experimental runs. The non-parametric Friedman test revealed significant differences among the models (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i> = 0.00013). SAINT and AutoInt consistently outperformed the other approaches, while ResNet and TT demonstrated the weakest performance. Post-hoc analysis using the Nemenyi test showed no statistically significant differences among mid-tier models (TabNet, GTT, NODE). A critical difference (CD) further confirmed SAINT and AutoInt as the most effective architectures for addressing complex, imbalanced educational data. These findings highlight the importance of aligning model selection and resampling techniques with the complexity of the task and the characteristics of the data.

Topics & Concepts