Assessing Reflective Writing on a Pediatric Clerkship by Using a Modified Bloom’s Taxonomy

• Published in: Ambulatory pediatrics : the official journal of the Ambulatory Pediatric Association 2007-07, Vol.7 (4), p.285-291
• Main Authors: Plack, Margaret M., PT, EdD, Driscoll, Maryanne, PhD, Marquez, Maria, MD, Cuppernull, Lynn, MS, Maring, Joyce, PT, EdD, Greenberg, Larrie, MD
• Format: Article
• Language: English
• Subjects: Adult; assessment; Bloom’s Taxonomy; Classification; Clinical Clerkship - methods; Clinical Clerkship - standards; Clinical Competence; Clinical medicine; Cognition - classification; Critical thinking; Decision making; District of Columbia; Educational evaluation; Educational Measurement - methods; Female; higher order thinking; Humans; Learning; Learning - classification; Male; Medical residencies; Observer Variation; Pediatrics; Pediatrics - education; Physicians; reflection; Skills; Student writing; Studies; Writing - standards
• ISSN: 1530-1567; 1876-2859; 1539-4409; 1876-2867
• DOI: 10.1016/j.ambp.2007.04.006

Objective Reflection enables learners to analyze their experiences and capture the wisdom that lies within. Effective teaching requires reliable methods of assessment. Several methods of assessing reflective writing have been described; however, they often require significant training, and reliability has seldom been assessed. This study was designed to determine the interrater reliability of a method of assessing reflective writing by using a modified Bloom’s Taxonomy. Methods Twenty-one third-year medical students maintained reflective journals throughout their pediatric clerkship. A coding schema based on Bloom’s Taxonomy was developed to assess the level of cognitive processing evident in the journals. Journals were independently assessed by 3 raters. Percent agreement, kappa statistics, and intraclass correlation coefficients (ICC [2,1]) were used to assess interrater reliability. Results Three hundred eight entries from 21 journals were assessed. Percent agreement ranged from 78.2% to 100%. Kappa statistic for each level ranged from 0.57 ± 0.04 to 0.73 ± 0.04, and for the highest level of processing evident it ranged from 0.52 ± .04 to 0.58 ± 0.04. ICC (2,1) for each level of cognitive processing ranged from 0.62 (F = 6.20; P = .000) to 1.00, and for the highest level of cognitive processing evident, it was 0.79 (F = 12.42; P = .000). Substantial to almost perfect agreement was attained. Conclusions Reflective journals allow learners to revisit their experiences for critical analysis and deeper learning. This study describes a reliable method, based on Bloom’s Taxonomy, of determining whether learners have achieved higher order thinking through reflective journal writing. This method can provide a baseline for facilitating higher order processing, critical thinking, and reflective practice.