Robust Estimation of Ability and Mental Speed Employing the Hierarchical Model for Responses and Response Times

• Published in: Journal of educational measurement 2021-09, Vol.58 (3), p.308-334
• Main Authors: Ranger, Jochen, Kuhn, Jörg‐Tobias, Wolgast, Anett
• Format: Article
• Language: English
• Subjects: Ability; Cognitive Processes; Educational Testing; Educational tests & measurements; Feedback (Response); Item Response Theory; Maximum likelihood method; Maximum Likelihood Statistics; Reaction Time; Response time; Robustness (Statistics); Simulation; Tests
• ISSN: 0022-0655; 1745-3984
• DOI: 10.1111/jedm.12284

Van der Linden's hierarchical model for responses and response times can be used in order to infer the ability and mental speed of test takers from their responses and response times in an educational test. A standard approach for this is maximum likelihood estimation. In real‐world applications, the data of some test takers might be partly irregular, resulting from rapid guessing or item preknowledge. The maximum likelihood estimator is not robust against contamination with irregular data. In this article, we propose a robust estimator of ability and mental speed. The estimator consists of two steps. In the first step, the mental speed is estimated with the estimator of Gervini and Yohai that ignores outlying response times. In the second step, the ability is estimated with an M‐estimator that down weights unusual responses given at unusual response times. This is achieved by combining the hard‐rejection weights of Gervini and Yohai with the M‐estimator suggested by Croux and Haesbroeck for the logistic regression model. The proposed estimator is consistent, almost as efficient as the maximum likelihood estimator in uncontaminated data and robust in contaminated data. The performance of the estimator is analyzed in a simulation study and an empirical example.