IRT Models for Learning With Item-Specific Learning Parameters

• Published in: Journal of educational and behavioral statistics 2023-12, Vol.48 (6), p.866-888
• Main Authors: Yu, Albert, Douglas, Jeffrey A.
• Format: Article
• Language: English
• Subjects: Adaptive Testing; Cognition & reasoning; Data Analysis; Difficulty Level; Formative Evaluation; Growth Models; Item Analysis; Item Response Theory; Learning; Markov Processes; Monte Carlo Methods; Sample Size; Test Items; Test Length
• ISSN: 1076-9986; 1935-1054
• DOI: 10.3102/10769986231193096

We propose a new item response theory growth model with item-specific learning parameters, or ISLP, and two variations of this model. In the ISLP model, either items or blocks of items have their own learning parameters. This model may be used to improve the efficiency of learning in a formative assessment. We show ways that the ISLP model’s learning parameters can be estimated in simulation using Markov chain Monte Carlo (MCMC), demonstrate a way that the model could be used in the context of adaptive item selection to increase the rate of learning, and estimate the learning parameters in an empirical data analysis using the ISLP. In the simulation studies, the one-parameter logistic model was used as the measurement model to generate random response data with various test lengths and sample sizes. Ability growth was modeled with a few variations of the ISLP model, and it was verified that the parameters were accurately recovered. Secondly, we generated data using the linear logistic test model with known Q-matrix structure for the item difficulties. Using a two-step procedure gave very comparable results for the estimation of the learning parameters even when item difficulties were unknown. The potential benefit of using an adaptive selection method in conjunction with the ISLP model was shown by comparing total improvement in the examinees’ ability parameter to two other methods of item selection that do not utilize this growth model. If the ISLP holds, adaptive item selection consistently led to larger improvements over the other methods. A real data application of the ISLP was given to illustrate its use in a spatial reasoning study designed to promote learning. In this study, interventions were given after each block of ten items to increase ability. Learning parameters were estimated using MCMC.