A Mixture Response Time Process Model for Aberrant Behaviors and Item Nonresponses

Abstract

In high-stakes, large-scale, standardized tests with certain time limits, examinees are likely to engage in either one of the three types of behavior (e.g., van der Linden & Guo, 2008 van der Linden, W. J., & Guo, F. (2008). Bayesian procedures for identifying aberrant response–time patterns in adaptive testing. Psychometrika, 73(3), 365–384. https://doi.org/10.1007/s11336-007-9046-8[Crossref], [Web of Science ®] , [Google Scholar]; Wang & Xu, 2015 Wang, C., & Xu, G. (2015). A mixture hierarchical model for response times and response accuracy. The British Journal of Mathematical and Statistical Psychology, 68(3), 456–477. https://doi.org/10.1111/bmsp.12054[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]): solution behavior, rapid guessing behavior, and cheating behavior. Oftentimes examinees do not always solve all items due to various reasons such as time limit or test-taking strategy. Item nonresponses may happen due to intentionally omitting some items (omitted responses) or due to lack of time (not-reached responses). Both types are related to latent abilities and hence the missingness is nonignorable. In this article, we proposed an innovative mixture response time process model (1) to detect two most common aberrant behaviors: rapid guessing behavior and cheating behavior, and (2) to account for two types of item nonresponses: not-reached items and omitted items. The new model combines the two-stage approach of Wang et al. (2018 Wang, C., Xu, G., & Shang, Z. (2018). A two-stage approach to differentiating normal and aberrant behavior in computer based testing. Psychometrika, 83(1), 223–254. https://doi.org/10.1007/s11336-016-9525-x[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]) with Lu and Wang (2020 Lu, J., & Wang, C. (2020). A response time process model for not-reached and omitted items. Journal of Educational Measurement, 57(4), 584–620. https://doi.org/10.1111/jedm.12270[Crossref], [Web of Science ®] , [Google Scholar]) model. It also contains two steps: (1) a mixture response time process model is first fitted to the responses and response times data to distinguish normal and aberrant behaviors and to account for the missing data mechanism; and (2) a Bayesian residual index is used to further distinguish rapid guessing and cheating behaviors. Simulation results show that the two-stage method yields accurate item and person parameter estimates, as well as high detection of aberrant behaviors. A real data analysis was conducted to illustrate the potential application of the proposed method.

Topics & Concepts