A General Method of Empirical Q-matrix Validation

In contrast to unidimensional item response models that postulate a single underlying proficiency, cognitive diagnosis models (CDMs) posit multiple, discrete skills or attributes, thus allowing CDMs to provide a finer-grained assessment of examinees’ test performance. A common component of CDMs for...

Full description

Saved in:
Bibliographic Details
Published in:Psychometrika 2016-06, Vol.81 (2), p.253-273
Main Authors: de la Torre, Jimmy, Chiu, Chia-Yi
Format: Article
Language:English
Subjects:
Adolescent
Algorithms
Assessment
Behavioral Science and Psychology
Child
Cognition
Educational Measurement
Educational Psychology
Feasibility Studies
Humanities
Humans
Inferences
Item Response Theory
Law
Methods
Models, Psychological
Models, Statistical
Psychology
Psychometrics
Quantitative psychology
Random variables
Reproducibility of Results
Resistance (Psychology)
Statistical Theory and Methods
Statistics as Topic
Statistics for Social Sciences
Test Items
Test Theory
Testing and Evaluation
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In contrast to unidimensional item response models that postulate a single underlying proficiency, cognitive diagnosis models (CDMs) posit multiple, discrete skills or attributes, thus allowing CDMs to provide a finer-grained assessment of examinees’ test performance. A common component of CDMs for specifying the attributes required for each item is the Q-matrix. Although construction of Q-matrix is typically performed by domain experts, it nonetheless, to a large extent, remains a subjective process, and misspecifications in the Q-matrix, if left unchecked, can have important practical implications. To address this concern, this paper proposes a discrimination index that can be used with a wide class of CDM subsumed by the generalized deterministic input, noisy “and” gate model to empirically validate the Q-matrix specifications by identifying and replacing misspecified entries in the Q-matrix. The rationale for using the index as the basis for a proposed validation method is provided in the form of mathematical proofs to several relevant lemmas and a theorem. The feasibility of the proposed method was examined using simulated data generated under various conditions. The proposed method is illustrated using fraction subtraction data.
ISSN:0033-3123
1860-0980
DOI:10.1007/s11336-015-9467-8