Should we extend the currency of cognitive ability test scores? Considerations from construct, equity, and psychometric perspectives in medical selection

• Published in: International journal of selection and assessment 2024-03, Vol.32 (1), p.138-148
• Main Authors: Chiavaroli, Neville, Le, Luc, Parker‐Newlyn, Lyndal, Pywell, Sean
• Format: Article
• Language: English
• Subjects: Ability tests; Cognitive ability; cognitive ability testing; equity; medical school selection; practice effects; Quantitative psychology; score currency; test resits; widening participation
• ISSN: 0965-075X; 1468-2389
• DOI: 10.1111/ijsa.12453

Scores on cognitive ability tests used in tertiary admissions contexts generally have very limited currency. This can have significant implications for prospective applicants to high‐demand courses which use cognitive tests as part of the selection process. In this paper we present both psychometric and non‐psychometric considerations regarding the score currency of ability tests, using GAMSAT (Graduate Medical School Admissions Test) as an example. We found that GAMSAT scores showed sufficient stability at the cohort level for institutions to be reasonably confident that a test score would continue to provide a valid representation of cognitive ability for up to a 5‐year period. However, candidates’ pre‐test preparation will influence whether it is in their interest to re‐sit a test even within an extended currency period. Practitioner points Cognitive ability tests used in tertiary admissions contexts generally have very limited score currency, typically 1 or 2 years. This limited period of currency means that many candidates may need to re‐sit the test if not initially successful, which can have significant and wide‐ranging implications for both candidates and institutions, especially in terms of diversity and equity. Our research on GAMSAT re‐sit scores suggest that cognitive ability tests with robust equating procedures can have sufficient stability at the cohort level for institutions to be reasonably confident that a test score will continue to provide a valid representation of cognitive ability for up to a 5‐year period. Nonetheless, the variability observed in the scores of some individual candidates means that it may sometimes be in the interests to re‐sit the GAMSAT despite the extended currency; this will be influenced by the extent of personal preparation with which candidates attempt their first sitting.