Lessons learned from designing an open-source automated feedback system for STEM education
Steffen Steinert, Lars Krupp, Karina E. Avila, Anke S. Janssen, Verena Ruf, David Dzsotjan, Christian De Schryver, Jakob Karolus, Stefan Ruzika, Karen Joisten, Paul Lukowicz, Jochen Kühn, Norbert Wehn, Stefan Küchemann
Abstract
Abstract As distance learning becomes increasingly important and artificial intelligence tools continue to advance, automated systems for individual learning have attracted significant attention. However, the scarcity of open-source online tools that are capable of providing personalized feedback has restricted the widespread implementation of research-based feedback systems. In this work, we present RATsApp, an open-source automated feedback system (AFS) that incorporates research-based features such as formative feedback. The system focuses on core STEM competencies such as mathematical competence, representational competence, and data literacy. It also allows lecturers to monitor students’ progress. RATsApp can be used at different levels of STEM education or research, as it allows the creation and customization of the educational content. We present a specific case of its implementation in higher education, where we report the results of a usability survey (N=64), using the technology acceptance model 2 (TAM2), to evaluate the user experience of undergraduate students. Our findings confirm the applicability of the TAM2 framework, revealing that factors such as the relevance to the course of study, output quality, and ease of use significantly influence the perceived usefulness. We also found a linear relation between the perceived usefulness and the intention to use, which in turn is a significant predictor of the frequency of use. Moreover, the formative feedback feature of RATsApp received positive feedback, indicating its potential as an educational tool. Furthermore, as an open-source platform, RATsApp encourages public contributions to its ongoing development, fostering a collaborative approach to improve educational tools.