Is a picture worth a thousand words: an analysis of the difficulty and discrimination parameters of illustrated vs. text-alone vignettes in histology multiple choice questions

• Published in: BMC medical education 2015-10, Vol.15 (1), p.184-184, Article 184
• Main Authors: Holland, Jane, O'Sullivan, Robin, Arnett, Richard
• Format: Article
• Language: English
• Subjects: Choice Behavior; Comprehension - physiology; Course Content; Curriculum; Difficulty Level; Education, Medical, Undergraduate - methods; Educational Measurement - methods; Educational Media; Effect Size; Epistemology; Equipment and supplies; Error of Measurement; Evidence; Female; Guidelines; Health education; High Achievement; Histology - education; Humans; Item Analysis; Learning Modules; Male; Measurement Techniques; Medical Education; Protocol Analysis; Psychometrics; Quality Assurance; Quality Control; Spatial Ability; Statistics, Nonparametric; Students, Medical - psychology; Teaching; Teaching Methods; Test Taking Skills; Test Theory; True Scores; Validity; Vignettes; Young Adult
• ISSN: 1472-6920; 1472-6920
• DOI: 10.1186/s12909-015-0452-9

Advances in cognitive load theory have led to greater understanding of how we process verbal and visual material during learning, but the evidence base with regard to the use of images within written assessments is still sparse. This study examines whether the inclusion of images within the stimulus format of multiple choice questions (MCQs) has a predictable or consistent influence on psychometric item properties, such as difficulty or discrimination. Item analysis data from three consecutive years of histology multiple choice examinations were included in this study. All items were reviewed and categorised according to whether their stem, or stimulus format, was purely textual or included an associated image. A total of 195 MCQs were identified for inclusion and analysed using classical test theory; 95 used text alone and 100 included an image within the question stem. The number of students per examination ranged from 277 to 347, with a total of 60,850 student-question interactions. We initially examined whether the inclusion of an image within the item stem altered the item difficulty using Mann-Whitney U. The median item difficulty for images with purely textual stems was 0.77, while that for items incorporating an appropriate image was 0.80; this difference was not significant (0.77 vs. 0.80; p = 0.862, Mann-Whitney-U = 4818.5). Mean values showed that the Item Discrimination Index appeared unaffected by the inclusion of an image within the stem, and Item point biserial correlation also showed no difference in means between these two groups (Independent samples t-test; 2-tailed). We demonstrate that the addition of illustrations within undergraduate histology Multiple Choice Question stems has no overall influence on item difficulty, or measures of item discrimination. We conclude that the use of images in this context is statistically uncritical, and suggest that their inclusion within item stems should be based upon the principles of constructive alignment. However, further research with respect to the effect of images within item stems on cognitive processing, particularly with regard to image complexity or type, would enable the development of more informed guidelines for their use.