Problematizing the concept of the "borderline" group in performance assessments

Introduction: Many standard setting procedures focus on the performance of the "borderline" group, defined through expert judgments by assessors. In performance assessments such as Objective Structured Clinical Examinations (OSCEs), these judgments usually apply at the station level. Metho...

Full description

Saved in:
Bibliographic Details
Published in:Medical teacher 2017-05, Vol.39 (5), p.469-475
Main Authors: Homer, Matt, Pell, Godfrey, Fuller, Richard
Format: Article
Language:English
Subjects:
Assessors
Candidates
Clinical assessment
Clinical Competence - standards
Compensation
Decision Making
Education, Medical, Undergraduate - methods
Education, Medical, Undergraduate - standards
Educational Measurement - methods
Evaluation
Humans
Standard Setting
Standard Setting (Scoring)
Students
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Introduction: Many standard setting procedures focus on the performance of the "borderline" group, defined through expert judgments by assessors. In performance assessments such as Objective Structured Clinical Examinations (OSCEs), these judgments usually apply at the station level. Methods and results: Using largely descriptive approaches, we analyze the assessment profile of OSCE candidates at the end of a five year undergraduate medical degree program to investigate the consistency of the borderline group across stations. We look specifically at those candidates who are borderline in individual stations, and in the overall assessment. While the borderline group can be clearly defined at the individual station level, our key finding is that the membership of this group varies considerably across stations. Discussion and conclusions: These findings pose challenges for some standard setting methods, particularly the borderline group and objective borderline methods. They also suggest that institutions should ensure appropriate conjunctive rules to limit compensation in performance between stations to maximize "diagnostic accuracy". In addition, this work highlights a key benefit of sequential testing formats in OSCEs. In comparison with a traditional, single-test format, sequential models allow assessment of "borderline" candidates across a wider range of content areas with concomitant improvements in pass/fail decision-making.
ISSN:0142-159X
1466-187X
DOI:10.1080/0142159X.2017.1296563