Simple techniques to bypass GenAI text detectors: implications for inclusive education

This study investigates the efficacy of six major Generative AI (GenAI) text detectors when confronted with machine-generated content modified to evade detection (n = 805). We compare these detectors to assess their reliability in identifying AI-generated text in educational settings, where they are...

Full description

Saved in:
Bibliographic Details
Published in:International Journal of Educational Technology in Higher Education 2024-12, Vol.21 (1), p.53-25, Article 53
Main Authors: Perkins, Mike, Roe, Jasper, Vu, Binh H., Postma, Darius, Hickerson, Don, McGaughran, James, Khuat, Huy Q.
Format: Article
Language:English
Subjects:
Academic integrity
Academic misconduct
Adversarial techniques
AI text detectors
Computer Appl. in Social and Behavioral Sciences
Computer Science
Computers and Education
Detectors
Educational evaluation
Educational Technology
Generative artificial intelligence
Higher Education
Humanities
Inclusive education
Information Systems Applications (incl.Internet)
Integrity
Law
Research Article
Statistics for Social Sciences
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This study investigates the efficacy of six major Generative AI (GenAI) text detectors when confronted with machine-generated content modified to evade detection (n = 805). We compare these detectors to assess their reliability in identifying AI-generated text in educational settings, where they are increasingly used to address academic integrity concerns. Results show significant reductions in detector accuracy (17.4%) when faced with simple techniques to manipulate the AI generated content. The varying performances of GenAI tools and detectors indicate they cannot currently be recommended for determining academic integrity violations due to accuracy limitations and the potential for false accusation which undermines inclusive and fair assessment practices. However, these tools may support learning and academic integrity when used non-punitively. This study aims to guide educators and institutions in the critical implementation of AI text detectors in higher education, highlighting the importance of exploring alternatives to maintain inclusivity in the face of emerging technologies.
ISSN:2365-9440
2365-9440
DOI:10.1186/s41239-024-00487-w