An Unsupervised‐Learning‐Based Approach to Compromised Items Detection

• Published in: Journal of educational measurement 2021-09, Vol.58 (3), p.413-433
• Main Authors: Pan, Yiqin, Wollack, James A.
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; Cheating; Cognitive style; Identification; Reaction time; Test Items
• ISSN: 0022-0655; 1745-3984
• DOI: 10.1111/jedm.12299

As technologies have been improved, item preknowledge has become a common concern in the test security area. The present study proposes an unsupervised‐learning‐based approach to detect compromised items. The unsupervised‐learning‐based compromised item detection approach contains three steps: (1) classify responses of each examinee as either normal or aberrant based on both the item response and the response time; (2) use a recursive algorithm to cluster examinees into groups based on their response similarity; (3) identify the group with strongest preknowledge signal and report questionable items as compromised. Results show that under the conditions studied, provided the amount of preknowledge is not overwhelming and aberrance effect is at least moderate, the approach controls the false‐negative rate at a relatively low level and the false‐positive rate at an extremely low level.