Using TextEvaluator® to Quantify Sources of Linguistic Complexity in Textbooks Targeted at First‐Grade Readers Over the Past Half Century

• Published in: ETS research report series 2015-12, Vol.2015 (2), p.1-17
• Main Authors: Sheehan, Kathleen M., Flor, Michael, Napolitano, Diane, Ramineni, Chaitanya
• Format: Article
• Language: English
• Subjects: Automated text complexity measurement; Common Core State Standards; readability
• ISSN: 2330-8516; 2330-8516
• DOI: 10.1002/ets2.12085

This paper considers whether the sources of linguistic complexity presented within texts targeted at 1st‐grade readers have increased, decreased, or held steady over the 52‐year period from 1962 to 2013. A collection of more than 450 texts is examined. All texts were selected from Grade 1 textbooks published by Scott Foresman during the targeted time period. Analyses are implemented using the TextEvaluator® tool, a comprehensive text complexity evaluation tool designed to help teachers, textbook publishers, and test developers identify and quantify text‐based sources of comprehension difficulty within informational, literary, and mixed texts. Results suggest that 1st‐grade textbooks published over the past half century have included an increasing proportion of informational passages, and this shift has been accompanied by the following specific changes: (a) an increase in the proportion of words that tend to appear less frequently in printed text, (b) an increase in the proportion of words that are more characteristic of academic text as opposed to fiction or conversation, (c) lower levels of referential cohesion, (d) lower levels of narrativity, and (e) fewer instances of an interactive/conversational style. These findings suggest that, in contrast to the claim of a “general, steady decline” in textbook complexity, text‐based sources of comprehension difficulty within Grade 1 texts have either risen or held steady throughout the past half century. Report Number: ETS RR–15–38