Cognitive Diagnostic Models for Random Guessing Behaviors
Chia‐Ling Hsu, Kuan‐Yu Jin, Ming Ming Chiu
Abstract
Many test-takers do not carefully answer every test question, so they quickly answer without thoughtful consideration (rapid guessing, RG). However, researchers have not modeled RG when assessing student learning with cognitive diagnostic models (CDMs) to personalize feedback on a set of fine-grained skills (or attributes). Therefore, this study proposes an advanced CDM with item response and response time to model RG and thereby enhance cognitive diagnosis. This study tests the parameter recovery of this new CDM with a series of simulations via Markov chain Monte Carlo methods in JAGS. Also, this study tests the degrees to which the standard and proposed CDMs fit the student response data for the Programme for International Student Assessment (PISA) 2015 computer-based mathematics test. This new CDM outperformed the simpler CDM that ignored RG; the new CDM showed less bias and greater precision for both item and person estimates, and greater classification accuracy of test results. Meanwhile, the empirical study showed different levels of student RG across test items and confirmed the findings in the simulations.