LSTM-SHAP based academic performance prediction for disabled learners in virtual learning environments: a statistical analysis approach

Abstract

Abstract With the increasing use of Virtual Learning Environments (VLEs), student performance prediction has become a necessary task to better academic outcomes, especially for disabled students who experience specific constraints. Machine learning (ML) models like Random Forest (RF), XGBoost, and Artificial Neural Networks (ANN) have shown high predictive accuracy. However, these models often lack interpretability and making it difficult for educators to understand the key factors influencing student success. In addition, studies only concentrate on general student populations and do not incorporate complete demographic and behavioural data in their analyses, making it difficult to address the needs of disabled students. To address these challenges, this study proposed a novel approach by combining Long Short-Term Memory (LSTM) networks with SHapley Additive exPlanations (SHAP). LSTM captured the temporal dependencies in student interaction data, and SHAP make the model interpretable by identifying the most influential features responsible for achieving good learning performance. The proposed system allows for prediction accuracy and transparent decision making, thereby facilitating targeted interventions for disabled students. The proposed model is tested on the OULAD dataset that contains demographic, engagement, and assessment-related information. The pre-processing techniques are applied to handle missing values, normalize numerical attributes, and balance class distributions. The experimental results demonstrate that the proposed LSTM-SHAP model outperforms traditional machine learning methods in predicting student performance, achieving an accuracy of 92.88%, precision of 92.96%, recall of 92.80%, and F1-score of 92.79% on the OULAD dataset. In a cross-domain evaluation, it maintains strong generalizability with 85.0% accuracy, 98.4% precision, 78.5% recall, and an F1-score of 87.2% when tested on the xAPI dataset. Additionally, SHAP analysis reveals that assessment scores, time spent on course materials, and forum interactions are the most important factors behind academic success. The statistical tests, such as ANOVA, Friedman tests and post hoc Nemenyi, are applied to assess the statistical significance of the differences in performance between the models, and results suggest that LSTM outperforms other models in each performance metric.

Topics & Concepts