Study on the Interrater Reliability of an OSPE (Objective Structured Practical Examination) - Subject to the Evaluation Mode in the Phantom Course of Operative Dentistry

The aim of the study presented here was to evaluate the reliability of an OSPE end-of-semester exam in the phantom course for operative dentistry in Frankfurt am Main taking into consideration different modes of evaluation (examiner's checklist versus instructor's manual) and number of exa...

Full description

Saved in:
Bibliographic Details
Published in:GMS journal for medical education 2016-01, Vol.33 (4), p.Doc61-Doc61
Main Authors: Schmitt, Laura, Möltner, Andreas, Rüttermann, Stefan, Gerhardt-Szép, Susanne
Format: Article
Language:English
Subjects:
Checklist
dentistry
Dentistry, Operative - education
Educational Measurement
evaluator
feedback
Humans
instructor's manual
OSCE
OSPE
Reproducibility of Results
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The aim of the study presented here was to evaluate the reliability of an OSPE end-of-semester exam in the phantom course for operative dentistry in Frankfurt am Main taking into consideration different modes of evaluation (examiner's checklist versus instructor's manual) and number of examiners (three versus four). In an historic, monocentric, comparative study, two different methods of evaluation were examined in a real end-of-semester setting held in OSPE form (Group I: exclusive use of an examiner's checklist versus Group II: use of an examiner's checklist including an instructor's manual). For the analysis of interrater reliability, the generalisability theory was applied that contains a generalisation of the concept of internal consistency (Cronbach's alpha). The results show that the exclusive use of the examiner's checklist led to higher interrater reliability values than the in-depth instructor's manual used in addition to the list. In summary it can be said that the examiner's checklists used in the present study, without the instructor's manual, resulted in the highest interrater reliability in combination with three evaluators within the context of the completed OSPE.
ISSN:2366-5017
2366-5017
DOI:10.3205/zma001060