Automated text scoring and real‐time adjustable feedback: Supporting revision of scientific arguments involving uncertainty

• Published in: Science education (Salem, Mass.) Mass.), 2019-05, Vol.103 (3), p.590-622
• Main Authors: Lee, Hee‐Sun, Pallant, Amy, Pryputniewicz, Sarah, Lord, Trudi, Mulholland, Matthew, Liu, Ou Lydia
• Format: Article
• Language: English
• Subjects: Ambiguity (Semantics); automated text scoring; Automation; Computer Assisted Testing; Earth Science; English Language Learners; Environmental Education; Feedback; Feedback (Response); Formative Evaluation; formative feedback; High School Students; Middle School Students; Persuasive Discourse; Pretests Posttests; Revision (Written Composition); Science Education; Science Process Skills; Science Tests; scientific argumentation; Scoring; Scoring Rubrics; Secondary School Students; Student Evaluation; Students; uncertainty; Water; Writing (Composition)
• ISSN: 0036-8326; 1098-237X
• DOI: 10.1002/sce.21504

This paper describes HASbot, an automated text scoring and real‐time feedback system designed to support student revision of scientific arguments. Students submit open‐ended text responses to explain how their data support claims and how the limitations of their data affect the uncertainty of their explanations. HASbot automatically scores these text responses and returns the scores with feedback to students. Data were collected from 343 middle‐ and high‐school students taught by nine teachers across seven states in the United States. A mixed methods design was applied to investigate (a) how students’ utilization of HASbot impacted their development of uncertainty‐infused scientific arguments; (b) how students used feedback to revise their arguments, and (c) how the current design of HASbot supported or hindered students’ revisions. Paired sample t tests indicate that students made significant gains from pretest to posttest in uncertainty‐infused scientific argumentation, ES = 1.52 SD, p