Towards AI-Generated Essay Classification Using Numerical Text Representation

The detection of essays written by AI compared to those authored by students is increasingly becoming a significant issue in educational settings. This research examines various numerical text representation techniques to improve the classification of these essays. Utilizing a diverse dataset, we un...

Full description

Saved in:
Bibliographic Details
Published in:Applied sciences 2024-11, Vol.14 (21), p.9795
Main Authors: Krawczyk, Natalia, Probierz, Barbara, Kozak, Jan
Format: Article
Language:English
Subjects:
Accuracy
Artificial intelligence
Classification
Computational linguistics
Datasets
Decision trees
Education
Feature selection
Language
Language processing
Large language models
Natural language
Natural language interfaces
natural language processing
Neural networks
numerical text representations
Regression analysis
Sensors
Social networks
Support vector machines
Text categorization
text classification
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The detection of essays written by AI compared to those authored by students is increasingly becoming a significant issue in educational settings. This research examines various numerical text representation techniques to improve the classification of these essays. Utilizing a diverse dataset, we undertook several preprocessing steps, including data cleaning, tokenization, and lemmatization. Our system analyzes different text representation methods such as Bag of Words, TF-IDF, and fastText embeddings in conjunction with multiple classifiers. Our experiments showed that TF-IDF weights paired with logistic regression reached the highest accuracy of 99.82%. Methods like Bag of Words, TF-IDF, and fastText embeddings achieved accuracies exceeding 96.50% across all tested classifiers. Sentence embeddings, including MiniLM and distilBERT, yielded accuracies from 93.78% to 96.63%, indicating room for further refinement. Conversely, pre-trained fastText embeddings showed reduced performance, with a lowest accuracy of 89.88% in logistic regression. Remarkably, the XGBoost classifier delivered the highest minimum accuracy of 96.24%. Specificity and precision were above 99% for most methods, showcasing high capability in differentiating between student-created and AI-generated texts. This study underscores the vital role of choosing dataset-specific text representations to boost classification accuracy.
ISSN:2076-3417
2076-3417
DOI:10.3390/app14219795