Leveraging Large Language Models for Precision Monitoring of Chemotherapy-Induced Toxicities: A Pilot Study with Expert Comparisons and Future Directions

• Published in: Cancers 2024, Vol.16 (16)
• Main Authors: Ruiz Sarrias, Oskitz, Martínez del Prado, María Purificación, Sala Gonzalez, María Ángeles, Azcuna Sagarduy, Josune, Casado Cuesta, Pablo, Figaredo Berjano, Covadonga, Galve-Calvo, Elena, López de San Vicente Hernández, Borja, López-Santillán, María, Nuño Escolástico, Maitane, Sánchez Togneri, Laura, Sande Sardina, Laura, Pérez Hoyos, María Teresa, Abad Villar, María Teresa, Zabalza Zudaire, Maialen, Sayar Beristain, Onintza
• Format: Report
• Language: English
• Subjects: Cancer; Chemotherapy; Complications and side effects; Drug utilization; Drugs; Health aspects; Technology application
• ISSN: 2072-6694; 2072-6694
• DOI: 10.3390/cancers16162830

This study evaluated the ability of Large Language Models (LLMs) to classify subjective toxicities from chemotherapy by comparing them with expert oncologists. Using fictitious cases, it was demonstrated that LLMs can achieve accuracy similar to that of oncologists in general toxicity categories, although they need improvement in specific categories. LLMs show great potential for enhancing patient monitoring and reducing the workload of doctors. Future research should focus on training LLMs specifically for medical tasks and validating these findings with real patients, always ensuring accuracy and ethical data management. Introduction: Large Language Models (LLMs), such as the GPT model family from OpenAI, have demonstrated transformative potential across various fields, especially in medicine. These models can understand and generate contextual text, adapting to new tasks without specific training. This versatility can revolutionize clinical practices by enhancing documentation, patient interaction, and decision-making processes. In oncology, LLMs offer the potential to significantly improve patient care through the continuous monitoring of chemotherapy-induced toxicities, which is a task that is often unmanageable for human resources alone. However, existing research has not sufficiently explored the accuracy of LLMs in identifying and assessing subjective toxicities based on patient descriptions. This study aims to fill this gap by evaluating the ability of LLMs to accurately classify these toxicities, facilitating personalized and continuous patient care. Methods: This comparative pilot study assessed the ability of an LLM to classify subjective toxicities from chemotherapy. Thirteen oncologists evaluated 30 fictitious cases created using expert knowledge and OpenAI’s GPT-4. These evaluations, based on the CTCAE v.5 criteria, were compared to those of a contextualized LLM model. Metrics such as mode and mean of responses were used to gauge consensus. The accuracy of the LLM was analyzed in both general and specific toxicity categories, considering types of errors and false alarms. The study’s results are intended to justify further research involving real patients. Results: The study revealed significant variability in oncologists’ evaluations due to the lack of interaction with fictitious patients. The LLM model achieved an accuracy of 85.7% in general categories and 64.6% in specific categories using mean evaluations with mild errors at 96.4% and se