Automated Scoring in Intelligent Tutoring Systems
Robert J. Mislevy, Duanli Yan, Janice D. Gobert, Michael São Pedro
Abstract
Simulation-based assessments open the door to a radically new paradigm of learning, characterized by interaction and personalized adaptation. We discuss automated scoring (AS) in intelligent tutoring systems (ITSs) that support students’ learning through their fairly unconstrained interactions in simulation environments. We highlight the necessary interplay among theories of cognition and learning in the domain and of the domain itself, simulation and interface design, and psychometric modeling, which must all be coordinated with AS from the start and not as an afterthought. We draw on concepts from evidence-centered design to explicate the relationships and illustrate the ideas with two examples: the Hydrive ITS to help US Airforce Force trainees learn to troubleshoot aircraft hydraulics systems and the Inq-ITS system to foster middle school students’ skills in science inquiry.