Accuracy of ChatGPT in Neurolocalization

Introduction ChatGPT (OpenAI Incorporated, Mission District, San Francisco, United States) is an artificial intelligence (AI) chatbot with advanced communication skills and a massive knowledge database. However, its application in medicine, specifically in neurolocalization, necessitates clinical re...

Full description

Saved in:
Bibliographic Details
Published in:Curēus (Palo Alto, CA) CA), 2024-04, Vol.16 (4), p.e59143-e59143
Main Authors: Dabbas, Waleed F, Odeibat, Yousef M, Alhazaimeh, Mohammad, Hiasat, Mohammad Y, Alomari, Amer A, Marji, Ala, Samara, Qais A, Ibrahim, Bilal, Al Arabiyat, Rashed M, Momani, Ghena
Format: Article
Language:English
Subjects:
Accuracy
Anatomy
Artificial intelligence
Chatbots
Datasets
Hypothalamus
Medical diagnosis
Neurosurgery
Self evaluation
Spinal cord
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Introduction ChatGPT (OpenAI Incorporated, Mission District, San Francisco, United States) is an artificial intelligence (AI) chatbot with advanced communication skills and a massive knowledge database. However, its application in medicine, specifically in neurolocalization, necessitates clinical reasoning in addition to deep neuroanatomical knowledge. This article examines ChatGPT's capabilities in neurolocalization. Methods Forty-six text-based neurolocalization case scenarios were presented to ChatGPT-3.5 from November 6th, 2023, to November 16th, 2023. Seven neurosurgeons evaluated ChatGPT's responses to these cases, utilizing a 5-point scoring system recommended by ChatGPT, to score the accuracy of these responses. Results ChatGPT-3.5 achieved an accuracy score of 84.8% in generating "completely correct" and "mostly correct" responses. ANOVA analysis suggested a consistent scoring approach between different evaluators. The mean length of the case text was 69.8 tokens (SD 20.8). Conclusion While this accuracy score is promising, it is not yet reliable for routine patient care. We recommend keeping interactions with ChatGPT concise, precise, and simple to improve response accuracy. As AI continues to evolve, it will hold significant and innovative breakthroughs in medicine.
ISSN:2168-8184
2168-8184
DOI:10.7759/cureus.59143