Automatic analysis of cognitive presence in online discussions: An approach using deep learning and explainable artificial intelligence

Abstract

This paper proposes the adoption of a deep learning method to automate the categorisation of online discussion messages according to the phases of cognitive presence, a fundamental construct from the widely used Community of Inquiry (CoI) framework of online learning. We investigated not only the performance of a deep learning classifier but also its generalisability and interpretability, using explainable artificial intelligence algorithms. In the study, we compared a Convolution Neural Network (CNN) model with the previous approaches reported on the literature based on random forest classifiers and linguistics features of psychological processes and cohesion. The CNN classifier trained and tested on the individual data set reached results up to Cohen's κ of 0.528, demonstrating a similar performance to those of the random forest classifiers. Also, the generalisability outcomes of the CNN classifiers across two disciplinary courses were similar to the results of the random forest approach. Finally, the visualisations of explainable artificial intelligence provide novel insights into identifying the phases of cognitive presence by word-level relevant indicators, as a complement to the feature importance analysis from the random forest. Thus, we envisage combining the deep learning method and the conventional machine learning algorithms (e.g. random forest) as complementary approaches to classify the phases of cognitive presence.

Topics & Concepts