A Nonparametric Composite Group DIF Index for Focal Groups Stemming from Multicategorical Variables

The purpose of this study is to develop a nonparametric DIF method that (a) compares focal groups directly to the composite group that will be used to develop the reported test score scale, and (b) allows practitioners to explore for DIF related to focal groups stemming from multicategorical variabl...

Full description

Saved in:
Bibliographic Details
Published in:Journal of educational measurement 2024-09, Vol.61 (3), p.432-457
Main Authors: Huggins‐Manley, Corinne, Raborn, Anthony W., Jones, Peggy K., Myers, Ted
Format: Article
Language:English
Subjects:
Achievement tests
Educational tests & measurements
Focus groups
Indexes
Nonparametric Statistics
Permutations
School districts
Simulation
Statistical significance
Variables
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The purpose of this study is to develop a nonparametric DIF method that (a) compares focal groups directly to the composite group that will be used to develop the reported test score scale, and (b) allows practitioners to explore for DIF related to focal groups stemming from multicategorical variables that constitute a small proportion of the overall testing population. We propose the nonparametric root expected proportion squared difference (REPSD) index that evaluates the statistical significance of composite group DIF for relatively small focal groups stemming from multicategorical focal variables, with decisions of statistical significance based on quasi‐exact p values obtained from Monte Carlo permutations of the DIF statistic under the null distribution. We conduct a simulation to evaluate conditions under which the index produces acceptable Type I error and power rates, as well as an application to a school district assessment. Practitioners can calculate the REPSD index in a freely available package we created in the R environment.
ISSN:0022-0655
1745-3984
DOI:10.1111/jedm.12394