Automated Feedback and Automated Scoring in the Elementary Grades: Usage, Attitudes, and Associations with Writing Outcomes in a Districtwide Implementation of MI Write

• Published in: International journal of artificial intelligence in education 2021-06, Vol.31 (2), p.234-276
• Main Authors: Wilson, Joshua, Huang, Yue, Palermo, Corey, Beard, Gaysha, MacArthur, Charles A.
• Format: Article
• Language: English
• Subjects: Academically Gifted; Artificial Intelligence; At Risk Students; Attitudes; Automation; Classroom Techniques; Computer Science; Computer Software; Computers and Education; Curricula; Education; Educational Facilities Improvement; Educational Technology; Elementary School Students; Elementary School Teachers; English Language Learners; Error Correction; Essay Tests; Essays; Feedback; Feedback (Response); Gifted; Grammar; Interpersonal Relationship; Learning; Middle School Teachers; Middle Schools; National Surveys; Peer review; Peers; Positive Attitudes; Predictor Variables; Program Implementation; Revision (Written Composition); School districts; Scores; Scoring; Sentence Structure; Skills; Software; Spelling; Standardized Tests; State Surveys; Student Attitudes; Student Motivation; Student writing; Students; Students with Disabilities; Teacher Attitudes; Teachers; User Interfaces and Human Computer Interaction; Writing Evaluation; Writing instruction; Writing Teachers
• ISSN: 1560-4292; 1560-4306
• DOI: 10.1007/s40593-020-00236-w

This study examined a naturalistic, districtwide implementation of an automated writing evaluation (AWE) software program called MI Write in elementary schools. We specifically examined the degree to which aspects of MI Write were implemented, teacher and student attitudes towards MI Write, and whether MI Write usage along with other predictors like demographics and writing self-efficacy explained variability in students’ performance on a proximal and distal measure of writing performance. The participants included 1935 students in Grades 3–5 and 135 writing teachers from 14 elementary schools in a mid-Atlantic school district. Findings indicated that though MI Write was somewhat under-utilized, teachers and students held positive attitudes towards the AWE system. Usage of MI Write had a mixed and limited predictive effect on outcomes: The number of essays written had a small predictive effect on state test performance for Grades 3 and 5; gain on revision had a moderate predictive effect on posttest writing quality and a small predictive effect for Grade 5 state test performance. Students’ average AWE scores showed consistently moderate to large predictive effects for all outcomes. Interpreted in light of the underlying architecture of MI Write, findings have implications for other school districts considering implementing AWE as well as the design of AWE systems intended to support the teaching and learning of writing.