Similarity measures in automated essay scoring systems: A ten-year review

In automated essay scoring (AES) systems, similarity techniques are used to compute the score for student answers. Several methods to compute similarity have emerged over the years. However, only a few of them have been widely used in the AES domain. This work shows the findings of a ten-year review...

Full description

Saved in:
Bibliographic Details
Published in:Education and information technologies 2022-05, Vol.27 (4), p.5573-5604
Main Authors: Ramnarain-Seetohul, Vidasha, Bassoo, Vandana, Rosunally, Yasmine
Format: Article
Language:English
Subjects:
Automation
Computation
Computational linguistics
Computer Appl. in Social and Behavioral Sciences
Computer Assisted Testing
Computer Science
Computers and Education
Education
Educational evaluation
Educational Technology
Educational tests & measurements
Essays
Information Seeking
Information Systems Applications (incl.Internet)
Language processing
Natural language interfaces
Scoring
Search Strategies
Standardized tests
Technology application
User Interfaces and Human Computer Interaction
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In automated essay scoring (AES) systems, similarity techniques are used to compute the score for student answers. Several methods to compute similarity have emerged over the years. However, only a few of them have been widely used in the AES domain. This work shows the findings of a ten-year review on similarity techniques applied in AES systems and discusses the efficiency and limitations of current methods. In the final review, thirty-four (34) articles published between 2010 and 2020 were included. The metrics used to evaluate the performance of the AES systems are also elaborated. The review was conducted using the Kitchenham method, whereby three research questions were formulated and a search strategy was developed. Research papers were chosen based on pre-defined inclusion and quality assessment criteria. This review has identified two types of similarity techniques used in AES systems. In addition, several methods were used to compute the score for student answers in the AES systems. The similarity computation in AES systems is dependent on several factors, hence many studies have combined multiple methods in a single system yielding good results. In addition, the review found that the quadratic weighted kappa (QWK) was most frequently used to evaluate AES systems.
ISSN:1360-2357
1573-7608
DOI:10.1007/s10639-021-10838-z