Almost Nobody Is Using ChatGPT to Write Academic Science Papers (Yet)

We assessed 19,000 scientific introductions to measure the level of undisclosed use of ChatGPT in scientific papers published in 2023 and early 2024. We applied a “stylistics” approach that has previously been shown to be effective at differentiating AI-generated text from human-written text in a va...

Full description

Saved in:
Bibliographic Details
Published in:Big data and cognitive computing 2024-10, Vol.8 (10), p.133
Main Authors: Desaire, Heather, Isom, Madeline, Hua, David
Format: Article
Language:English
Subjects:
academic publication
AI text detection
Artificial intelligence
Authorship
Automation
Chatbots
ChatGPT
Citations
Disclosure
Editing
generative AI
Generative artificial intelligence
Machine learning
MDPI
Plagiarism
Writing
XGBoost
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We assessed 19,000 scientific introductions to measure the level of undisclosed use of ChatGPT in scientific papers published in 2023 and early 2024. We applied a “stylistics” approach that has previously been shown to be effective at differentiating AI-generated text from human-written text in a variety of venues. Ten different MDPI journals were selected for this study, and the rate of use of undisclosed AI writing in these journals was fairly consistent across the journals. We estimate that ChatGPT was used for writing or significant editing in about 1 to 3% of the introductions tested. This analysis is the first systematic study of detecting undisclosed ChatGPT in published manuscripts in cases where obvious indicators, such as phrases like “regenerate response”, are not present. The work demonstrates that generative AI is not polluting mainstream journals to any appreciable extent and that the overwhelming majority of scientists remain hesitant to embrace this tool for late-stage writing and editing.
ISSN:2504-2289
2504-2289
DOI:10.3390/bdcc8100133