An Unsupervised‐Learning‐Based Approach to Compromised Items Detection

As technologies have been improved, item preknowledge has become a common concern in the test security area. The present study proposes an unsupervised‐learning‐based approach to detect compromised items. The unsupervised‐learning‐based compromised item detection approach contains three steps: (1) c...

Full description

Saved in:
Bibliographic Details
Published in:Journal of educational measurement 2021-09, Vol.58 (3), p.413-433
Main Authors: Pan, Yiqin, Wollack, James A.
Format: Article
Language:English
Subjects:
Artificial Intelligence
Cheating
Cognitive style
Identification
Reaction time
Test Items
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:As technologies have been improved, item preknowledge has become a common concern in the test security area. The present study proposes an unsupervised‐learning‐based approach to detect compromised items. The unsupervised‐learning‐based compromised item detection approach contains three steps: (1) classify responses of each examinee as either normal or aberrant based on both the item response and the response time; (2) use a recursive algorithm to cluster examinees into groups based on their response similarity; (3) identify the group with strongest preknowledge signal and report questionable items as compromised. Results show that under the conditions studied, provided the amount of preknowledge is not overwhelming and aberrance effect is at least moderate, the approach controls the false‐negative rate at a relatively low level and the false‐positive rate at an extremely low level.
ISSN:0022-0655
1745-3984
DOI:10.1111/jedm.12299