Do assessor comments on a multi-source feedback instrument provide learner-centred feedback?

• Published in: Medical education 2013-11, Vol.47 (11), p.1080-1088
• Main Authors: Vivekananda-Schmidt, Pirashanthie, MacKillop, Lucy, Crossley, Jim, Wade, Winnie
• Format: Article
• Language: English
• Subjects: Attitude of Health Personnel; Biological and medical sciences; Clinical Competence; Education, Medical - methods; Feedback; Health participants; Health Personnel; Humans; Learning; Medical sciences; Pilot Projects; Public health. Hygiene; Public health. Hygiene-occupational medicine; Reproducibility of Results
• ISSN: 0308-0110; 1365-2923
• DOI: 10.1111/medu.12249

Objectives Free‐text comments in multi‐source feedback are intended to facilitate change in the assessee's practice. This study was designed to utilise a large dataset of free‐text comments obtained in a national pilot study in order to investigate how helpful these free‐text comments may be to assessees. Methods We investigated: (i) which areas of performance are usually addressed by free‐text comments; (ii) to what extent assessors' (doctors, nurses, allied health professionals and clerical or managerial staff) comments correspond to assessees' (career‐grade doctors) self‐assessments, and (iii) whether the comments contain specific behavioural evidence and suggestions for change. Initially comments were read through to identify commonly recurring themes. A strong theme was ‘respondent‐centredness’, which refers to the extent to which comments focus on issues that are of value to the assessor rather than to the assessee's personal development. In response to this, the data were re‐evaluated against predefined research questions to assess how constructive comments were for the assessee's personal development. Results Of 11 483 assessor forms, 4777 (42%) included free‐text comments. A total of 513 forms contained at least one below average score and 286 (56%) of these forms contained the assessor's free‐text feedback. Free‐text comments were mostly rater‐centred and addressed the effect of the assessee on the colleague's working life rather than areas of relevance to the assessee's personal development. A total of 1806 assessor/assessee pairs of comments were compared; most demonstrated clear differences of opinion or interpretation. Reliability and supportiveness were over‐represented; clinical performance and personal development were under‐represented. The comments were unlikely to provide specific behavioural evidence or to address how change might be initiated. Conclusions Our data indicate that, in their current form, the overwhelming majority of free‐text comments add little to facilitate improvement in assessees' personal development and performance. Discuss ideas arising from the article at ‘discuss’