Graduate instructors navigating the AI frontier: The role of ChatGPT in higher education

• Published in: Computers and education open 2024-06, Vol.6, p.100166, Article 100166
• Main Authors: Parker, Luke, Carter, Chris, Karakas, Alice, Loper, A. Jane, Sokkar, Ahmad
• Format: Article
• Language: English
• Subjects: AI use in assessments; Artificial intelligence; Chatgpt; Comparative analysis of assessment; Higher education; Performance chatgpt vs. real-life students; Plagiarism
• ISSN: 2666-5573; 2666-5573
• DOI: 10.1016/j.caeo.2024.100166

•Performance of ChatGPT in assessment Generation: ChatGPT outperformed the average student in seven classes, securing a grades or higher in 10 out of 11 instances. It produced top-quality assessments, ranking highly in 80 % of the cases across these classes.•GTA accuracy in identifying AI-Generated Assessments: graduate teaching assistants (GTAs) accurately identified 50 % of AI-generated assessments. Notably, GTAs showed higher accuracy in detecting more complex, advanced AI-generated assessments compared to basic, introductory ones. This increased accuracy could be attributed to the more detailed responses referencing components outside what was expected of undergraduate students making the AI-generated responses in these areas more distinguishable.•ChatGPT's adaptability across subject Areas: ChatGPT demonstrated remarkable adaptability across various subjects, proficiently creating assessments for diverse disciplines including Technology, TESOL, introduction to Teaching, english Methods, and Mathematics. While it showed varying degrees of strength across these areas, ChatGPT consistently performed well, indicating its robust capability in handling a wide range of assessment types and academic subjects.•Potential impact of class size on GTA Accuracy: within the finding there appears to be a correlation between class size and GTA accuracy in identifying AI-generated work, with higher accuracy of instructors being able to detect AI-generated work in smaller classes compared to a lower accuracy level in larger classes.•Implications for AI in assessment Practices: the study underscores AI's potential in being used within higher education assessment. It suggests that AI-generated assessments can yield high-quality results. Further research is needed to explore factors like instructor characteristics, class size, and the effectiveness of AI detection tools such as turnitin. This research study explores the use of artificial intelligence (AI) in undergraduate assessments, specifically focusing on the ability of graduate teaching assistants (GTAs) to identify AI-generated assessments and the performance of ChatGPT, an AI model, in producing high-quality work. The study examines four guiding research questions and hypotheses related to the accuracy of GTA identification, the achievement of AI-generated work compared to student marks, the impact of GTA characteristics on identification accuracy, and the variation in identification and assessment across different