Analyzing the Efficacy of ECoach in Supporting Gateway Course Success Through Tailored Support
Rebecca L. Matz, Kyle W. Schulz, Elizabeth Hanley, Holly A. Derry, Benjamin T. Hayward, Benjamin P. Koester, Caitlin Hayward, Timothy A. McKay
Abstract
Large courses act as gateways for college students and often have poor outcomes, particularly in STEM fields where the pace of improvement has been glacial. Students encounter barriers to persistence like low grades, competitive cultures, and a lack of motivation and belonging. Tailored technology systems offer one promising path forward. In this observational study, we report on the use of one such system, called ECoach, that provides students resources based on their psychosocial profile, performance metrics, and pattern of ECoach usage. We investigated ECoach efficacy in five courses enrolling 3,599 students using a clustering method to group users by engagement level and subsequent regression analyses. We present results showing significant positive relationships with small effect sizes between ECoach engagement and final course grade as well as grade anomaly, a performance measure that takes into account prior course grades. The courses with the strongest relationship between ECoach engagement and performance offered nominal extra credit incentives yet show improved grades well above this “investment” from instructors. Such small incentives may act as a catalyst that spurs deeper engagement with the platform. The impact of specific ECoach features and areas for future study are discussed.