Effects of DIF in MST routing in ILSAs

The advance and access to technology have allowed for the implementation of adaptive testing in international large-scale assessments (ILSAs). Multistage testing (MST) in ILSAs offers opportunities and advantages compared to linear testing. However, when dozens of highly heterogeneous systems partic...

Full description

Saved in:
Bibliographic Details
Published in:Large-scale assessments in education 2023-12, Vol.11 (1), p.22-22, Article 22
Main Authors: Valdivia Medinaceli, Montserrat, Rutkowski, Leslie, Svetina Valdivia, Dubravka, Rutkowski, David
Format: Article
Language:English
Subjects:
Adaptive Testing
Assessment
Differential item functioning
Education
Educational Policy and Politics
Educational tests & measurements
Humanities
International education
International large-scale assessment
Law
Methodology
Multistage testing
Simulation
Statistics for Social Sciences
Testing and Evaluation
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The advance and access to technology have allowed for the implementation of adaptive testing in international large-scale assessments (ILSAs). Multistage testing (MST) in ILSAs offers opportunities and advantages compared to linear testing. However, when dozens of highly heterogeneous systems participate in ILSAs, the sources of heterogeneity pose challenges to any cross-cultural measurement endeavor. With the recent implementation of adaptive designs in ILSAs, little is known about differential item functioning (DIF) effects, mainly when an MST design is used in an ILSA context. Through a simulation study grounded on the empirical basis of the Trends in International Mathematics and Science Study (TIMSS) data, this paper examines the impact of DIF in MST routing in ILSA under different routing strategies. Results showed that Merit routing is highly accurate even when the amount and magnitude of DIF is high, whereas suboptimal routing showed poor accuracy across DIF conditions. As expected, Merit routing has better proficiency recovery parameters than a suboptimal routing mechanism. Implications and recommendations for test developers are included in the discussion section.
ISSN:2196-0739
2196-0739
DOI:10.1186/s40536-023-00165-9