Psychometric validation of the Laval developmental benchmarks scale for family medicine

Background With the implementation of competency-based education in family medicine, there is a need for summative end-of-rotation assessments that are criterion-referenced rather than normative. Laval University's family residency program therefore developed the Laval Developmental Benchmarks...

Full description

Saved in:
Bibliographic Details
Published in:BMC medical education 2021-06, Vol.21 (1), p.1-357, Article 357
Main Authors: Renaud, Jean-Sébastien, Lacasse, Miriam, Côté, Luc, Théorêt, Johanne, Rheault, Christian, Simard, Caroline
Format: Article
Language:English
Subjects:
Behavioral Objectives
Benchmarking
Clinical competence
Competence
Competency Based Education
Computation
Content Validity
Criterion-referenced assessment
Criterion-referenced tests
Evaluation
Evaluators
Evidence
Family medicine
Graduate Medical Education
High Achievement
Investigations
Medical education
Medicine
Psychometrics
Quantitative psychology
Questionnaires
Rating Scales
Scale models
Study and teaching
Supervision
Teacher Selection
Teaching Methods
Validation
Validity
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Background With the implementation of competency-based education in family medicine, there is a need for summative end-of-rotation assessments that are criterion-referenced rather than normative. Laval University's family residency program therefore developed the Laval Developmental Benchmarks Scale for Family Medicine (DBS-FM), based on competency milestones. This psychometric validation study investigates its internal structure and its relation to another variable, two sources of validity evidence. Methods We used assessment data from a cohort of residents (n = 1432 assessments) and the Rasch Rating Scale Model to investigate its reliability, dimensionality, rating scale functioning, targeting of items to residents' competency levels, biases (differential item functioning), items hierarchy (adequacy of milestones ordering), and score responsiveness. Convergent validity was estimated by its correlation with the clinical rotation decision (pass, in difficulty/fail). Results The DBS-FM can be considered as a unidimensional scale with good reliability for non-extreme scores (.83). The correlation between expected and empirical items hierarchies was of .78, p < .0001.Year 2 residents achieved higher scores than year 1 residents. It was associated with the clinical rotation decision. Conclusion Advancing its validation, this study found that the DBS-FM has a sound internal structure and demonstrates convergent validity. Keywords: Criterion-referenced assessment, Validation, Family medicine
ISSN:1472-6920
1472-6920
DOI:10.1186/s12909-021-02797-3