Three versions of an atopic dermatitis case report written by humans, artificial intelligence, or both: Identification of authorship and preferences

• Published in: The journal of allergy and clinical immunology. Global 2025-02, Vol.4 (1), p.100373, Article 100373
• Main Authors: Giavina Bianchi, Mara, D’adario, Andrew, Giavina Bianchi, Pedro, Machado, Birajara Soares, Agondi, Rosana, Almeida, Stephanie K.A., Alves Junior, Wandilson Xavier, Armelin, Larissa M., Vivolo Aun, Marcelo, Bordignon, Natália, Boufleur, Karla, Brunheroto, Felipe B., Callegaro, Elisabeth A., Castro, Paula Lazaretti M., Chong-Neto, Herberto Jose, Dall’Osto, Mariana D., Abou Dias, Julia, Ferreira, Viviane Heintze, Oliveira Feodrippe, André Luiz, Fonseca, Livia G., Garcia, Clydia M., Giavina-Bianchi, Bruna H., Goudouris, Ekaterini, Gonçalves, Danilo Gois, Hernandes, Debora D., Imad, Malek, Izabel, Larissa S., Cauê Jacintho, Lucas, Khouri-Panzarin, Carolina, Kuschnir, Fabio, Pádua Lima, Maria Beatriz, Lopes, Amanda I., Macêdo Nóbrega Lopes, Larissa Nathalia, Rocha de Magalhães, Alice, Mansour, Eli, Marinho, Ana Karolina B.B., Martimiano, Vivian S., Milori, Pedro H., Marcondes Mutarelli, Antonio, Gonçalves Nogueira, Guilherme Paes, Oguido, Beatriz K.T., Alarcon de Oliveira, Bruna S., Costa de Oliveira, Emerson, Padulla, Georgia A., D’Ordaz Lhano Santos, Letícia, Meneses Santos, Micaelly Samara, Sarinho, Emanuel, Schoen, Marcela, Sousa, Brian Lucas A., Magalhães de Souza-Lima, Eduardo, Todt, Beatriz C., Braz da Silva Vaz, Najla
• Format: Article
• Language: English
• Subjects: artificial intelligence; ChatGPT; Generative Pre-training Transformer (GPT); large language model (LLM); medical survey; scientific writing
• ISSN: 2772-8293; 2772-8293
• DOI: 10.1016/j.jacig.2024.100373

The use of artificial intelligence (AI) in scientific writing is rapidly increasing, raising concerns about authorship identification, content quality, and writing efficiency. This study investigates the real-world impact of ChatGPT, a large language model, on those aspects in a simulated publication scenario. Forty-eight individuals representing 3 medical expertise levels (medical students, residents, and experts in allergy or dermatology) evaluated 3 blinded versions of an atopic dermatitis case report: one each human written (HUM), AI generated (AI), and combined written (COM). The survey assessed authorship, ranked their preference, and graded 13 quality criteria for each text. Time taken to generate each manuscript was also recorded. Authorship identification accuracy mirrored the odds at 33%. Expert participants (50.9%) demonstrated significantly higher accuracy compared to residents (27.7%) and students (19.6%, P < .001). Participants favored AI-assisted versions (AI and COM) over HUM (P < .001), with COM receiving the highest quality scores. COM and AI achieved 83.8% and 84.3% reduction in writing time, respectively, compared to HUM, while showing 13.9% (P < .001) and 11.1% improvement in quality (P < .001), respectively. However, experts assigned the lowest score for the references of the AI manuscript, potentially hindering its publication. AI can deceptively mimic human writing, particularly for less experienced readers. Although AI-assisted writing is appealing and offers significant time savings, human oversight remains crucial to ensure accuracy, ethical considerations, and optimal quality. These findings underscore the need for transparency in AI use and highlight the potential of human-AI collaboration in the future of scientific writing.