Leveraging the Innovation-Based Learning Framework to Predict and Understand Student Success in Innovation
Lauren Singelmann, Daniel Ewert
Abstract
As more educational data becomes available, learning analytics and educational data mining are allowing instructors to make data-driven decisions like never before. Much of the work in learning analytics and educational data mining has been done in traditional subjects like math and reading, but these tools also have the potential to shape our understanding of the complexity of engineering design and innovation. This work uses learning analytics and educational data mining to analyze and interpret student data collected in a course where students work on innovation projects. Specifically, this work improves previous classification models developed for innovation in classroom settings by implementing the Innovation-Based Learning (IBL) Framework – a qualitative model that sorts student actions into seven categories. The models developed include 1) a text classifier that automatically groups student text into the categories of the IBL framework with higher reliability than a team of human raters ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\kappa =0.760$ </tex-math></inline-formula> versus <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\kappa =0.627$ </tex-math></inline-formula> ), 2) a text classifier that uses the IBL framework to predict student performance on new cohorts better than previously developed models (ROC AUC =0.710 versus ROC AUC =0.640), and 3) a quantitative classifier that uses the IBL framework to differentiate between lower and higher performing students better than previously developed models (ROC AUC =0.782 versus ROC AUC =0.491). Not only does this work detail the development and testing of these models, but it also uses the results to suggest best practices of teaching innovation in educational settings.