Automatic Content Analysis of Online Discussions for Cognitive Presence: A Study of the Generalizability Across Educational Contexts
Valter dos Santos Mendonça Neto, Vitor Rolim, Anderson Pinheiro, Rafael Dueire Lins, Dragan Gašević, Rafael Ferreira Mello
Abstract
This article investigates the impact of educational contexts on automatic classification of online discussion messages according to cognitive presence, an essential construct of the community of inquiry model. In particular, the work reported in the article analyzed online discussion messages written in Brazilian Portuguese from two different courses that were from different subject areas (biology and technology) and had different teaching presence in the online discussions. The study explored a set of 127 features of online discussion messages and a random forest classifier to automatically recognize the phases of the cognitive presence in online discussion messages. The results showed that the classifier achieved better performance when applied to the entire dataset. It reveals that when a classifier is created for a specific course it is not generic enough to be applied to a course from a different field of knowledge. The results also showed the importance of the features that were predictive of the phases of the cognitive presence in the educational context. Based on the findings of this study, future work should adopt the same feature set as used in the current study, but it should train the classifier of the cognitive presence on datasets in subject areas related to the topic of the discussions.