IoT-based Multimodal Analysis for Smart Education: Current Status, Challenges and Opportunities
Wenbin Gan, Minh Son Dao, Koji Zettsu, Yuan Sun
Abstract
IoT-based multimodal learning analytics promises to obtain an in-depth understanding of the learning process. It provides the insights for not only the explicit learning indicators but also the implicit attributes of learners, based on which further potential learning support can be timely provided in both physical and cyber world accordingly. In this paper, we present a systematic review of the existing studies for examining the empirical evidences on the usage of IoT data in education and the capabilities of multimodal analysis to provide useful insights for smarter education. In particular, we classify the multimodal data into four categories based on the data sources (data from digital, physical, physiological and environmental spaces). Moreover, we propose a concept framework for better understanding the current state of the filed and summarize the insights into six main themes (learner behavior understanding, learner affection computing, smart learning environment, learning performance prediction, group collaboration modeling and intelligent feedback) based on the objectives for intelligent learning. The associations between different combinations of data modalities and various learning indicators are comprehensively discussed. Finally, the challenges and future directions are also presented from three aspects.