A novel workplace-based assessment for competency-based decisions and learner feedback

Background: Increased recognition of the importance of competency-based education and assessment has led to the need for practical and reliable methods to assess relevant skills in the workplace. Methods: A novel milestone-based workplace assessment system was implemented in 15 pediatrics residency...

Full description

Saved in:
Bibliographic Details
Published in:Medical teacher 2018-11, Vol.40 (11), p.1143-1150
Main Authors: Hicks, Patricia J., Margolis, Melissa J., Carraccio, Carol L., Clauser, Brian E., Donnelly, Kathleen, Fromme, H. Barrett, Gifford, Kimberly A., Poynter, Sue E., Schumacher, Daniel J., Schwartz, Alan
Format: Article
Language:English
Subjects:
Clinical skills
Competence
Competency Based Education
Evaluation
Feedback
Generalizability
Generalizability Theory
Multisource
Observation
Pediatrics
Reliability
Workplaces
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Background: Increased recognition of the importance of competency-based education and assessment has led to the need for practical and reliable methods to assess relevant skills in the workplace. Methods: A novel milestone-based workplace assessment system was implemented in 15 pediatrics residency programs. The system provided: (1) web-based multisource feedback (MSF) and structured clinical observation (SCO) instruments that could be completed on any computer or mobile device; and (2) monthly feedback reports that included competency-level scores and recommendations for improvement. Results: For the final instruments, an average of five MSF and 3.7 SCO assessment instruments were completed for each of 292 interns; instruments required an average of 4-8 min to complete. Generalizability coefficients >0.80 were attainable with six MSF observations. Users indicated that the new system added value to their existing assessment program; the need to complete the local assessments in addition to the new assessments was identified as a burden of the overall process. Conclusions: Outcomes - including high participation rates and high reliability compared to what has traditionally been found with workplace-based assessment - provide evidence for the validity of scores resulting from this novel competency-based assessment system. The development of this assessment model is generalizable to other specialties.
ISSN:0142-159X
1466-187X
DOI:10.1080/0142159X.2018.1461204