Modeling Test-Taking Non-effort in MIRT Models

• Published in: Frontiers in psychology 2019-02, Vol.10, p.145-145
• Main Authors: Liu, Yue, Li, Zhen, Liu, Hongyun, Luo, Fang
• Format: Article
• Language: English
• Subjects: effort-moderated model; missing data; multidimensional item response theory; Psychology; response time; test-taking effort
• ISSN: 1664-1078; 1664-1078
• DOI: 10.3389/fpsyg.2019.00145

The validity of inferences based on test scores will be threatened when examinees' test-taking non-effort is ignored. A possible solution is to add test-taking effort indicators in the measurement model after the non-effortful responses are flagged. As a new application of the multidimensional item response theory (MIRT) model for non-ignorable missing responses, this article proposed a MIRT method to account for non-effortful responses. Two simulation studies were conducted to examine the impact of non-effortful responses on item and latent ability parameter estimates, and to evaluate the performance of the MIRT method, comparing to the three-parameter logistic (3PL) model as well as the effort-moderated model. Results showed that: (a) as the percentage of non-effortful responses increased, the unidimensional 3PL model yielded poorer parameter estimates; (b) the MIRT model could obtain as accurate item parameter estimates as the effort-moderated model; (c) the MIRT model provided the most accurate ability parameter estimates when the correlation between test-taking effort and ability was high. A real data analysis was also conducted for illustration. The limitation and future research were discussed further.