A Special Case of Brennan's Index for Tests That Aim to Select a Limited Number of Students: A Monte Carlo Simulation Study

• Published in: Educational measurement, issues and practice issues and practice, 2022-12, Vol.41 (4), p.35-49
• Main Authors: Arikan, Serkan, Aybek, Eren Can
• Format: Article
• Language: English
• Subjects: Academic Ability; Accuracy; Classical test theory; Correlation; Difficulty Level; Educational tests & measurements; High Achievement; high‐stake tests; Individual Differences; Item Analysis; item discrimination indices; Item Response Theory; Monte Carlo Methods; Monte Carlo simulation; Selection Criteria; Simulation; Students; Test Items; Test Length; Test Use
• ISSN: 0731-1745; 1745-3992
• DOI: 10.1111/emip.12528

Many scholars compared various item discrimination indices in real or simulated data. Item discrimination indices, such as item‐total correlation, item‐rest correlation, and IRT item discrimination parameter, provide information about individual differences among all participants. However, there are tests that aim to select a very limited number of students, examinees, or candidates for allocated schools and job positions. Thus, there is a need to evaluate the performances of CTT and IRT item discrimination indices when the test purpose is to select a limited number of students. The purpose of the current Monte Carlo study is to evaluate item discrimination indices in the case of selecting a limited number of high‐achieving students. The results showed that a special case of Brennan's index, B10–90, provided more accurate information for this specific test purpose. Additionally, the effects of various factors, such as test length, ability distributions of examinees, and item difficulty variance on item discrimination indices were investigated. The performance of each item discrimination index is discussed in detail.