Adapting cognitive diagnosis computerized adaptive testing item selection rules to traditional item response theory

Currently, there are two predominant approaches in adaptive testing. One, referred to as cognitive diagnosis computerized adaptive testing (CD-CAT), is based on cognitive diagnosis models, and the other, the traditional CAT, is based on item response theory. The present study evaluates the performan...

Full description

Saved in:
Bibliographic Details
Published in:PloS one 2020-01, Vol.15 (1), p.e0227196-e0227196
Main Authors: Sorrel, Miguel A, Barrada, Juan R, de la Torre, Jimmy, Abad, Francisco José
Format: Article
Language:English
Subjects:
Accuracy
Algorithms
Bayes Theorem
Bias
Biology and Life Sciences
Cats
Cognition
Cognitive ability
Computer Simulation
Computers
Data Accuracy
Diagnosis
Educational Measurement - methods
Environmental impact
Humans
Integrals
Item response theory
Mathematical models
Medical diagnosis
Methods
Parameters
Performance evaluation
Physical Sciences
Psychometrics - methods
Quantitative psychology
Research and Analysis Methods
Security
Social psychology
Social Sciences
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Currently, there are two predominant approaches in adaptive testing. One, referred to as cognitive diagnosis computerized adaptive testing (CD-CAT), is based on cognitive diagnosis models, and the other, the traditional CAT, is based on item response theory. The present study evaluates the performance of two item selection rules (ISRs) originally developed in the CD-CAT framework, the double Kullback-Leibler information (DKL) and the generalized deterministic inputs, noisy "and" gate model discrimination index (GDI), in the context of traditional CAT. The accuracy and test security associated with these two ISRs are compared to those of the point Fisher information and weighted KL using a simulation study. The impact of the trait level estimation method is also investigated. The results show that the new ISRs, particularly DKL, could be used to improve the accuracy of CAT. Better accuracy for DKL is achieved at the expense of higher item overlap rate. Differences among the item selection rules become smaller as the test gets longer. The two CD-CAT ISRs select different types of items: items with the highest possible a parameter with DKL, and items with the lowest possible c parameter with GDI. Regarding the trait level estimator, expected a posteriori method is generally better in the first stages of the CAT, and converges with the maximum likelihood method when a medium to large number of items are involved. The use of DKL can be recommended in low-stakes settings where test security is less of a concern.
ISSN:1932-6203
1932-6203
DOI:10.1371/journal.pone.0227196