A Unified Comparison of IRT‐Based Effect Sizes for DIF Investigations

Several marginal effect size (ES) statistics suitable for quantifying the magnitude of differential item functioning (DIF) have been proposed in the area of item response theory; for instance, the Differential Functioning of Items and Tests (DFIT) statistics, signed and unsigned item difference in t...

Full description

Saved in:
Bibliographic Details
Published in:Journal of educational measurement 2023-06, Vol.60 (2), p.318-350
Main Author: Chalmers, R. Philip
Format: Article
Language:English
Subjects:
Definitions
Educational tests & measurements
Efficacy
Inferences
Item Response Theory
Monte Carlo Methods
Monte Carlo simulation
Recovery
Simulation
Statistical Analysis
Test Bias
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c3237-5e39f541dc66d520040fbf6ffe8e9dfeb7368ea0aa53c8748eff97d578617d983
cites cdi_FETCH-LOGICAL-c3237-5e39f541dc66d520040fbf6ffe8e9dfeb7368ea0aa53c8748eff97d578617d983
container_end_page 350
container_issue 2
container_start_page 318
container_title Journal of educational measurement
container_volume 60
creator Chalmers, R. Philip
description Several marginal effect size (ES) statistics suitable for quantifying the magnitude of differential item functioning (DIF) have been proposed in the area of item response theory; for instance, the Differential Functioning of Items and Tests (DFIT) statistics, signed and unsigned item difference in the sample statistics (SIDS, UIDS, NSIDS, and NUIDS), the standardized indices of impact, and the differential response functioning (DRF) statistics. However, the relationship between these proposed statistics has not been fully discussed, particularly with respect to population parameter definitions and recovery performance across independent samples. To address these issues, this article provides a unified presentation of competing DIF ES definitions and estimators, and evaluates the recovery efficacy of these competing estimators using a set of Monte Carlo simulation experiments. Statistical and inferential properties of the estimators are discussed, as well as future areas of research in this model‐based area of bias quantification.
doi_str_mv 10.1111/jedm.12347
format article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2822697120</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ericid>EJ1380024</ericid><sourcerecordid>2822697120</sourcerecordid><originalsourceid>FETCH-LOGICAL-c3237-5e39f541dc66d520040fbf6ffe8e9dfeb7368ea0aa53c8748eff97d578617d983</originalsourceid><addsrcrecordid>eNp9kM1KAzEURoMoWKsb90LAnTA1P5NJZlnbaZ2iCNquQzqTSEo7qUmr1JWP4DP6JKaOuPRu7uI7fPdyADjHqIfjXC90vephQlN-ADqYpyyhuUgPQQchQhKUMXYMTkJYIIQZZ7gDxn04a6yxuoYDt1orb4NroDOwfJx-fXzeqBCTwhhdbeCTfdcBGufhsBzBsnnVYWOf1ca6JpyCI6OWQZ_97i6YjYrp4Da5exiXg_5dUlFCecI0zQ1LcV1lWc0IQikyc5PFfqHz2ug5p5nQCinFaCV4KrQxOa8ZFxnmdS5oF1y2vWvvXrbxAblwW9_Ek5IIQrKcY4IiddVSlXcheG3k2tuV8juJkdyLkntR8kdUhC9aWHtb_YHFBFMRraUxx23-Zpd690-TnBTD-7bzGzDjc58</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2822697120</pqid></control><display><type>article</type><title>A Unified Comparison of IRT‐Based Effect Sizes for DIF Investigations</title><source>Applied Social Sciences Index &amp; Abstracts (ASSIA)</source><source>Wiley</source><source>ERIC</source><creator>Chalmers, R. Philip</creator><creatorcontrib>Chalmers, R. Philip</creatorcontrib><description>Several marginal effect size (ES) statistics suitable for quantifying the magnitude of differential item functioning (DIF) have been proposed in the area of item response theory; for instance, the Differential Functioning of Items and Tests (DFIT) statistics, signed and unsigned item difference in the sample statistics (SIDS, UIDS, NSIDS, and NUIDS), the standardized indices of impact, and the differential response functioning (DRF) statistics. However, the relationship between these proposed statistics has not been fully discussed, particularly with respect to population parameter definitions and recovery performance across independent samples. To address these issues, this article provides a unified presentation of competing DIF ES definitions and estimators, and evaluates the recovery efficacy of these competing estimators using a set of Monte Carlo simulation experiments. Statistical and inferential properties of the estimators are discussed, as well as future areas of research in this model‐based area of bias quantification.</description><identifier>ISSN: 0022-0655</identifier><identifier>EISSN: 1745-3984</identifier><identifier>DOI: 10.1111/jedm.12347</identifier><language>eng</language><publisher>Madison: Wiley</publisher><subject>Definitions ; Educational tests &amp; measurements ; Efficacy ; Inferences ; Item Response Theory ; Monte Carlo Methods ; Monte Carlo simulation ; Recovery ; Simulation ; Statistical Analysis ; Test Bias</subject><ispartof>Journal of educational measurement, 2023-06, Vol.60 (2), p.318-350</ispartof><rights>2022 by the National Council on Measurement in Education.</rights><rights>2023 by the National Council on Measurement in Education.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c3237-5e39f541dc66d520040fbf6ffe8e9dfeb7368ea0aa53c8748eff97d578617d983</citedby><cites>FETCH-LOGICAL-c3237-5e39f541dc66d520040fbf6ffe8e9dfeb7368ea0aa53c8748eff97d578617d983</cites><orcidid>0000-0001-5332-2810</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,27924,27925,30999</link.rule.ids><backlink>$$Uhttp://eric.ed.gov/ERICWebPortal/detail?accno=EJ1380024$$DView record in ERIC$$Hfree_for_read</backlink></links><search><creatorcontrib>Chalmers, R. Philip</creatorcontrib><title>A Unified Comparison of IRT‐Based Effect Sizes for DIF Investigations</title><title>Journal of educational measurement</title><description>Several marginal effect size (ES) statistics suitable for quantifying the magnitude of differential item functioning (DIF) have been proposed in the area of item response theory; for instance, the Differential Functioning of Items and Tests (DFIT) statistics, signed and unsigned item difference in the sample statistics (SIDS, UIDS, NSIDS, and NUIDS), the standardized indices of impact, and the differential response functioning (DRF) statistics. However, the relationship between these proposed statistics has not been fully discussed, particularly with respect to population parameter definitions and recovery performance across independent samples. To address these issues, this article provides a unified presentation of competing DIF ES definitions and estimators, and evaluates the recovery efficacy of these competing estimators using a set of Monte Carlo simulation experiments. Statistical and inferential properties of the estimators are discussed, as well as future areas of research in this model‐based area of bias quantification.</description><subject>Definitions</subject><subject>Educational tests &amp; measurements</subject><subject>Efficacy</subject><subject>Inferences</subject><subject>Item Response Theory</subject><subject>Monte Carlo Methods</subject><subject>Monte Carlo simulation</subject><subject>Recovery</subject><subject>Simulation</subject><subject>Statistical Analysis</subject><subject>Test Bias</subject><issn>0022-0655</issn><issn>1745-3984</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>7SW</sourceid><sourceid>7QJ</sourceid><recordid>eNp9kM1KAzEURoMoWKsb90LAnTA1P5NJZlnbaZ2iCNquQzqTSEo7qUmr1JWP4DP6JKaOuPRu7uI7fPdyADjHqIfjXC90vephQlN-ADqYpyyhuUgPQQchQhKUMXYMTkJYIIQZZ7gDxn04a6yxuoYDt1orb4NroDOwfJx-fXzeqBCTwhhdbeCTfdcBGufhsBzBsnnVYWOf1ca6JpyCI6OWQZ_97i6YjYrp4Da5exiXg_5dUlFCecI0zQ1LcV1lWc0IQikyc5PFfqHz2ug5p5nQCinFaCV4KrQxOa8ZFxnmdS5oF1y2vWvvXrbxAblwW9_Ek5IIQrKcY4IiddVSlXcheG3k2tuV8juJkdyLkntR8kdUhC9aWHtb_YHFBFMRraUxx23-Zpd690-TnBTD-7bzGzDjc58</recordid><startdate>20230601</startdate><enddate>20230601</enddate><creator>Chalmers, R. Philip</creator><general>Wiley</general><general>Wiley Subscription Services, Inc</general><scope>7SW</scope><scope>BJH</scope><scope>BNH</scope><scope>BNI</scope><scope>BNJ</scope><scope>BNO</scope><scope>ERI</scope><scope>PET</scope><scope>REK</scope><scope>WWN</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7QJ</scope><orcidid>https://orcid.org/0000-0001-5332-2810</orcidid></search><sort><creationdate>20230601</creationdate><title>A Unified Comparison of IRT‐Based Effect Sizes for DIF Investigations</title><author>Chalmers, R. Philip</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c3237-5e39f541dc66d520040fbf6ffe8e9dfeb7368ea0aa53c8748eff97d578617d983</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Definitions</topic><topic>Educational tests &amp; measurements</topic><topic>Efficacy</topic><topic>Inferences</topic><topic>Item Response Theory</topic><topic>Monte Carlo Methods</topic><topic>Monte Carlo simulation</topic><topic>Recovery</topic><topic>Simulation</topic><topic>Statistical Analysis</topic><topic>Test Bias</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Chalmers, R. Philip</creatorcontrib><collection>ERIC</collection><collection>ERIC (Ovid)</collection><collection>ERIC</collection><collection>ERIC</collection><collection>ERIC (Legacy Platform)</collection><collection>ERIC( SilverPlatter )</collection><collection>ERIC</collection><collection>ERIC PlusText (Legacy Platform)</collection><collection>Education Resources Information Center (ERIC)</collection><collection>ERIC</collection><collection>CrossRef</collection><collection>Applied Social Sciences Index &amp; Abstracts (ASSIA)</collection><jtitle>Journal of educational measurement</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Chalmers, R. Philip</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><ericid>EJ1380024</ericid><atitle>A Unified Comparison of IRT‐Based Effect Sizes for DIF Investigations</atitle><jtitle>Journal of educational measurement</jtitle><date>2023-06-01</date><risdate>2023</risdate><volume>60</volume><issue>2</issue><spage>318</spage><epage>350</epage><pages>318-350</pages><issn>0022-0655</issn><eissn>1745-3984</eissn><abstract>Several marginal effect size (ES) statistics suitable for quantifying the magnitude of differential item functioning (DIF) have been proposed in the area of item response theory; for instance, the Differential Functioning of Items and Tests (DFIT) statistics, signed and unsigned item difference in the sample statistics (SIDS, UIDS, NSIDS, and NUIDS), the standardized indices of impact, and the differential response functioning (DRF) statistics. However, the relationship between these proposed statistics has not been fully discussed, particularly with respect to population parameter definitions and recovery performance across independent samples. To address these issues, this article provides a unified presentation of competing DIF ES definitions and estimators, and evaluates the recovery efficacy of these competing estimators using a set of Monte Carlo simulation experiments. Statistical and inferential properties of the estimators are discussed, as well as future areas of research in this model‐based area of bias quantification.</abstract><cop>Madison</cop><pub>Wiley</pub><doi>10.1111/jedm.12347</doi><tpages>33</tpages><orcidid>https://orcid.org/0000-0001-5332-2810</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 0022-0655
ispartof Journal of educational measurement, 2023-06, Vol.60 (2), p.318-350
issn 0022-0655
1745-3984
language eng
recordid cdi_proquest_journals_2822697120
source Applied Social Sciences Index & Abstracts (ASSIA); Wiley; ERIC
subjects Definitions
Educational tests & measurements
Efficacy
Inferences
Item Response Theory
Monte Carlo Methods
Monte Carlo simulation
Recovery
Simulation
Statistical Analysis
Test Bias
title A Unified Comparison of IRT‐Based Effect Sizes for DIF Investigations
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-02T16%3A22%3A59IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20Unified%20Comparison%20of%20IRT%E2%80%90Based%20Effect%20Sizes%20for%20DIF%20Investigations&rft.jtitle=Journal%20of%20educational%20measurement&rft.au=Chalmers,%20R.%20Philip&rft.date=2023-06-01&rft.volume=60&rft.issue=2&rft.spage=318&rft.epage=350&rft.pages=318-350&rft.issn=0022-0655&rft.eissn=1745-3984&rft_id=info:doi/10.1111/jedm.12347&rft_dat=%3Cproquest_cross%3E2822697120%3C/proquest_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c3237-5e39f541dc66d520040fbf6ffe8e9dfeb7368ea0aa53c8748eff97d578617d983%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2822697120&rft_id=info:pmid/&rft_ericid=EJ1380024&rfr_iscdi=true