A Technical Note on IRT Simulation Studies: Dealing With Truth, Estimates, Observed Data, and Residuals

Simulation studies are extremely common in the item response theory (IRT) research literature. This article presents a didactic discussion of “truth” and “error” in IRT‐based simulation studies. We ultimately recommend that future research focus less on the simple recovery of parameters from a conve...

Full description

Saved in:
Bibliographic Details
Published in:Educational measurement, issues and practice issues and practice, 2018-09, Vol.37 (3), p.65-76
Main Authors: Luecht, Richard, Ackerman, Terry A.
Format: Article
Language:English
Subjects:
Comparative Analysis
Data
Error of Measurement
estimation
Ethics
Item Response Theory
model fit
Simulation
simulations
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Simulation studies are extremely common in the item response theory (IRT) research literature. This article presents a didactic discussion of “truth” and “error” in IRT‐based simulation studies. We ultimately recommend that future research focus less on the simple recovery of parameters from a convenient generating IRT model, and more on practical comparative estimation studies when the data are intentionally generated to incorporate nuisance dimensionality and other sources of nuanced contamination encountered with real data. A new framework is also presented for conceptualizing and comparing various residuals in IRT studies. The new framework allows even very different calibration and scoring IRT models to be compared on a common, convenient, and highly interpretable number‐correct metric. Some illustrative examples are included.
ISSN:0731-1745
1745-3992
DOI:10.1111/emip.12185