A Statistical Test for the Detection of Item Compromise Combining Responses and Response Times

A test of item compromise is presented which combines the test takers' responses and response times (RTs) into a statistic defined as the number of correct responses on the item for test takers with RTs flagged as suspicious. The test has null and alternative distributions belonging to the well...

Full description

Saved in:
Bibliographic Details
Published in:Journal of educational measurement 2023-06, Vol.60 (2), p.235-254
Main Authors: van der Linden, Wim J., Belov, Dmitry I.
Format: Article
Language:English
Subjects:
Alternative Assessment
Educational tests & measurements
Feedback (Response)
Item Analysis
Item Response Theory
Reaction Time
Response Style (Tests)
Response time
Test Items
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A test of item compromise is presented which combines the test takers' responses and response times (RTs) into a statistic defined as the number of correct responses on the item for test takers with RTs flagged as suspicious. The test has null and alternative distributions belonging to the well‐known family of compound binomial distributions, is simple to calculate, and has results that are easy to interpret. It also demonstrated nearly perfect power for the detection of compromise with no more than 10 test takers with preknowledge of the more difficult and discriminating items in a set of empirical examples. For the easier and less discriminating items, the presence of some 20 test takers with preknowledge still sufficed. A test based on the reverse statistic of the total time by test takers with responses flagged as suspicious may seem a natural alternative but misses the property of a monotone likelihood ratio necessary to decide between a test that should be left or right sided.
ISSN:0022-0655
1745-3984
DOI:10.1111/jedm.12346