Cognitive Diagnosis for Small Educational Programs: The General Nonparametric Classification Method

The focus of cognitive diagnosis (CD) is on evaluating an examinee’s strengths and weaknesses in terms of cognitive skills learned and skills that need study. Current methods for fitting CD models (CDMs) work well for large-scale assessments, where the data of hundreds or thousands of examinees are...

Full description

Saved in:
Bibliographic Details
Published in:Psychometrika 2018-06, Vol.83 (2), p.355-375
Main Authors: Chiu, Chia-Yi, Sun, Yan, Bian, Yanhong
Format: Article
Language:English
Subjects:
Assessment
Behavioral Science and Psychology
Classification
Cognition & reasoning
Cognitive ability
Data Analysis
Data processing
Diagnosis
Humanities
Law
Learning Processes
Nonparametric statistics
Psychology
Psychometrics
Statistical Theory and Methods
Statistics for Social Sciences
Testing and Evaluation
Thinking Skills
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The focus of cognitive diagnosis (CD) is on evaluating an examinee’s strengths and weaknesses in terms of cognitive skills learned and skills that need study. Current methods for fitting CD models (CDMs) work well for large-scale assessments, where the data of hundreds or thousands of examinees are available. However, the development of CD-based assessment tools that can be used in small-scale test settings, say, for monitoring the instruction and learning process at the classroom level has not kept up with the rapid pace at which research and development proceeded for large-scale assessments. The main reason is that the sample sizes of the small-scale test settings are simply too small to guarantee the reliable estimation of item parameters and examinees’ proficiency class membership. In this article, a general nonparametric classification (GNPC) method that allows for assigning examinees to the correct proficiency classes with a high rate when sample sizes are at the classroom level is proposed as an extension of the nonparametric classification (NPC) method (Chiu and Douglas in J Classif 30:225–250, 2013 ). The proposed method remedies the shortcomings of the NPC method and can accommodate any CDM. The theoretical justification and the empirical studies are presented based on the saturated general CDMs, supporting the legitimacy of using the GNPC method with any CDM. The results from the simulation studies and real data analysis show that the GNPC method outperforms the general CDMs when samples are small.
ISSN:0033-3123
1860-0980
DOI:10.1007/s11336-017-9595-4