Leveraging large language models to improve patient education on dry eye disease

• Published in: Eye (London) 2024-12
• Main Authors: Dihan, Qais A, Brown, Andrew D, Chauhan, Muhammad Z, Alzein, Ahmad F, Abdelnaem, Seif E, Kelso, Sean D, Rahal, Dania A, Park, Royce, Ashraf, Mohammadali, Azzam, Amr, Morsi, Mahmoud, Warner, David B, Sallam, Ahmed B, Saeed, Hajirah N, Elhusseiny, Abdelrahman M
• Format: Article
• Language: English
• ISSN: 0950-222X; 1476-5454; 1476-5454
• DOI: 10.1038/s41433-024-03476-5

Dry eye disease (DED) is an exceedingly common diagnosis in patients, yet recent analyses have demonstrated patient education materials (PEMs) on DED to be of low quality and readability. Our study evaluated the utility and performance of three large language models (LLMs) in enhancing and generating new patient education materials (PEMs) on dry eye disease (DED). We evaluated PEMs generated by ChatGPT-3.5, ChatGPT-4, Gemini Advanced, using three separate prompts. Prompts A and B requested they generate PEMs on DED, with Prompt B specifying a 6th-grade reading level, using the SMOG (Simple Measure of Gobbledygook) readability formula. Prompt C asked for a rewrite of existing PEMs at a 6th-grade reading level. Each PEM was assessed on readability (SMOG, FKGL: Flesch-Kincaid Grade Level), quality (PEMAT: Patient Education Materials Assessment Tool, DISCERN), and accuracy (Likert Misinformation scale). All LLM-generated PEMs in response to Prompt A and B were of high quality (median DISCERN = 4), understandable (PEMAT understandability ≥70%) and accurate (Likert Score=1). LLM-generated PEMs were not actionable (PEMAT Actionability