Exploring the potential uses of value-added metrics in the context of postgraduate medical education

• Published in: Medical teacher 2016-10, Vol.38 (10), p.1003-1010
• Main Authors: Gregory, Simon, Patterson, Fiona, Baron, Helen, Knight, Alec, Walsh, Kieran, Irish, Bill, Thomas, Sally
• Format: Article
• Language: English
• Subjects: Accountability; Clinical Competence; Cost analysis; Data analysis; Education, Medical, Graduate - standards; Educational Measurement - methods; Family physicians; General Practitioners - education; General Practitioners - standards; Graduate studies; Health education; Humans; Least Squares Statistics; Licensing; Licensing Examinations (Professions); Licensure, Medical; Medical education; Models, Educational; Postgraduate medical education; Problem solving; Regional variations; Regression Analysis; Trainees; United Kingdom; Value; Value added
• ISSN: 0142-159X; 1466-187X
• DOI: 10.3109/0142159X.2016.1147534

Context: Increasing pressure is being placed on external accountability and cost efficiency in medical education and training internationally. We present an illustrative data analysis of the value-added of postgraduate medical education. Method: We analysed historical selection (entry) and licensure (exit) examination results for trainees sitting the UK Membership of the Royal College of General Practitioners (MRCGP) licensing examination (N = 2291). Selection data comprised: a clinical problem solving test (CPST); a situational judgement test (SJT); and a selection centre (SC). Exit data was an applied knowledge test (AKT) from MRCGP. Ordinary least squares (OLS) regression analyses were used to model differences in attainment in the AKT based on performance at selection (the value-added score). Results were aggregated to the regional level for comparisons. Results: We discovered significant differences in the value-added score between regional training providers. Whilst three training providers confer significant value-added, one training provider was significantly lower than would be predicted based on the attainment of trainees at selection. Conclusions: Value-added analysis in postgraduate medical education potentially offers useful information, although the methodology is complex, controversial, and has significant limitations. Developing models further could offer important insights to support continuous improvement in medical education in future.