Can Patients With Urogenital Cancer Rely on Artificial Intelligence Chatbots for Treatment Decisions?

• Published in: Clinical genitourinary cancer 2024-12, Vol.22 (6), p.102206, Article 102206
• Main Authors: Erkan, Anil, Koc, Akif, Barali, Deniz, Satir, Atilla, Zengin, Salim, Kilic, Metin, Dundar, Gokce, Guzelsoy, Muhammet
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; ChatGPT; Comprehension; Female; Humans; Internet; Male; Patient Education as Topic - methods; Quality assessment; Reproducibility of Results; Treatment; Urogenital neoplasms; Urogenital Neoplasms - therapy
• ISSN: 1558-7673; 1938-0682; 1938-0682
• DOI: 10.1016/j.clgc.2024.102206

•Artificial intelligence Chatbots (AIC) provide moderate quality, low reliability, and very difficult to read information about the treatment of urogenital cancers.•AIC do not offer stage-specific treatment options, choosing the patient's treatment based on this information may lead to inaccurate results.•After accumulating literature revealing the shortcomings of AICs, they will eventually be able to address these issues independently. In the era of artificial intelligence, almost half of the patients use the internet to get information about their diseases. Our study aims to demonstrate the reliability of the information provided by artificial intelligence chatbots (AICs) about urogenital cancer treatments. The most frequently searched keyword about prostate, bladder, kidney, and testicular cancer treatment via Google Trends was asked to 3 different AICs (ChatGPT, Gemini, Copilot). The answers were evaluated by 5 different examiners in terms of readability, understandability, actionability, reliability, and transparency. The DISCERN score evaluation indicates that ChatGPT and Gemini provided moderate quality information, while Copilot's quality was low. (Total DISCERN scores; 41, 42, 35, respectively). PEMAT-P Understandability scores were low (40%) and PEMAT-P Actionability scores were moderate only for Gemini (60%) and low for the others (40%). Their readability according to the Coleman-Liau index was above the college level (16.9, 17.2, 16, respectively). In the era of artificial intelligence, patients will inevitably use AICs due to their easy and fast accessibility. However, patients need to recognize that AICs do not provide stage-specific treatment options, but only moderate-quality, low-reliability information about the disease, as well as information that is very difficult to read. We evaluated how reliable information about urogenital cancer treatment is provided by 3 different AICs, ChatGPT and Gemini provide moderate quality information and CoPilot provides low quality information. In addition, all AICs have low readability and do not offer stage-specific treatment options.