Comparing Traditional Versus Retrospective Pre-/Post-assessment in an Interdisciplinary Leadership Training Program

• Published in: Maternal and child health journal 2019-02, Vol.23 (2), p.191-200
• Main Authors: Thomas, Erin Vinoski, Wells, Rebecca, Baumann, Stephanie D., Graybill, Emily, Roach, Andrew, Truscott, Stephen D., Crenshaw, Mark, Crimmins, Daniel
• Format: Article
• Language: English
• Subjects: Bias; Cohort Studies; Educational Measurement - methods; Gynecology; Health Personnel - education; Health Personnel - statistics & numerical data; Humans; Interdisciplinary aspects; Interdisciplinary Studies; Leadership; Leadership training; Maternal and Child Health; Medical personnel training; Medicine; Medicine & Public Health; Methods; Patient Care Team - statistics & numerical data; Pediatrics; Population Economics; Professional Competence - standards; Professional Competence - statistics & numerical data; Program Evaluation - methods; Program Evaluation - standards; Program Evaluation - statistics & numerical data; Public Health; Retrospective Studies; Self evaluation; Self-Assessment; Sociology; Southeastern United States; Surveys and Questionnaires; Teaching - statistics & numerical data; Teaching - trends; Training
• ISSN: 1092-7875; 1573-6628
• DOI: 10.1007/s10995-018-2615-x

Objectives As the U.S. healthcare system shifts toward collaboration, demand for leaders with interdisciplinary skills increases. Leadership competencies guide interdisciplinary training programs; however, identifying cost-effective methods for evaluating leadership competencies is challenging, particularly when interdisciplinary trainees have different areas of expertise and professional goals. Traditional pre-/post-testing, a common method for evaluating leadership competencies, is subject to response-shift bias, which can occur when participants’ understanding of a construct changes between pre- and post-test. As a result, participants may rate their knowledge of the construct lower at post-test. Retrospective pre-tests are one method thought to reduce response-shift bias in pre-/post-tests. The current study explores the use of a retrospective pre-test to control for response-shift bias in an interdisciplinary training program. Methods Over three cohort years, thirty-four trainees from an interdisciplinary leadership program completed a self-assessment aligned with MCH leadership competencies. The traditional pre-test self-assessment was completed at the beginning of the training program. The retrospective pre-/post-test self-assessment was completed at the end of the training program. Results Retrospective pre/post-test scores indicate significant self-reported increases in all 24 leadership areas (p ≤ .001). Furthermore, participants’ self-ratings were significantly higher on the traditional pre-test for all 24 areas than on the retrospective pre-test (p ≤ .001). Conclusions for Practice Retrospective pre-tests appeared to control for response-shift bias and may be a cost-effective way to evaluate trainee change within an interdisciplinary leadership program. These findings suggest the methodology’s usefulness in interdisciplinary training and its potential use in the broader world of community-based MCH training initiatives.