Leveraging large language models to improve patient education on dry eye disease

Dry eye disease (DED) is an exceedingly common diagnosis in patients, yet recent analyses have demonstrated patient education materials (PEMs) on DED to be of low quality and readability. Our study evaluated the utility and performance of three large language models (LLMs) in enhancing and generatin...

Full description

Saved in:
Bibliographic Details
Published in:Eye (London) 2024-12
Main Authors: Dihan, Qais A, Brown, Andrew D, Chauhan, Muhammad Z, Alzein, Ahmad F, Abdelnaem, Seif E, Kelso, Sean D, Rahal, Dania A, Park, Royce, Ashraf, Mohammadali, Azzam, Amr, Morsi, Mahmoud, Warner, David B, Sallam, Ahmed B, Saeed, Hajirah N, Elhusseiny, Abdelrahman M
Format: Article
Language:English
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites cdi_FETCH-LOGICAL-c1351-2c0efb855b2f99eb898b6235748a845cec49d15e5f3d95fef9e230f05bd6293
container_end_page
container_issue
container_start_page
container_title Eye (London)
container_volume
creator Dihan, Qais A
Brown, Andrew D
Chauhan, Muhammad Z
Alzein, Ahmad F
Abdelnaem, Seif E
Kelso, Sean D
Rahal, Dania A
Park, Royce
Ashraf, Mohammadali
Azzam, Amr
Morsi, Mahmoud
Warner, David B
Sallam, Ahmed B
Saeed, Hajirah N
Elhusseiny, Abdelrahman M
description Dry eye disease (DED) is an exceedingly common diagnosis in patients, yet recent analyses have demonstrated patient education materials (PEMs) on DED to be of low quality and readability. Our study evaluated the utility and performance of three large language models (LLMs) in enhancing and generating new patient education materials (PEMs) on dry eye disease (DED). We evaluated PEMs generated by ChatGPT-3.5, ChatGPT-4, Gemini Advanced, using three separate prompts. Prompts A and B requested they generate PEMs on DED, with Prompt B specifying a 6th-grade reading level, using the SMOG (Simple Measure of Gobbledygook) readability formula. Prompt C asked for a rewrite of existing PEMs at a 6th-grade reading level. Each PEM was assessed on readability (SMOG, FKGL: Flesch-Kincaid Grade Level), quality (PEMAT: Patient Education Materials Assessment Tool, DISCERN), and accuracy (Likert Misinformation scale). All LLM-generated PEMs in response to Prompt A and B were of high quality (median DISCERN = 4), understandable (PEMAT understandability ≥70%) and accurate (Likert Score=1). LLM-generated PEMs were not actionable (PEMAT Actionability
doi_str_mv 10.1038/s41433-024-03476-5
format article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_3146912667</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3146912667</sourcerecordid><originalsourceid>FETCH-LOGICAL-c1351-2c0efb855b2f99eb898b6235748a845cec49d15e5f3d95fef9e230f05bd6293</originalsourceid><addsrcrecordid>eNo9kE1LAzEQhoMotlb_gAfJ0ctqvndzlGJVKCjowVvIbibLyn7UZLfQf29sqzDMvDDvOwwPQteU3FHCi_soqOA8I0xkhItcZfIEzeleCClO0ZxoSTLG2OcMXcT4RUha5uQczbhWBc0pnaO3NWwh2Lrpa9zaUEPqfT3ZJLrBQRvxOOCm24RhC3hjxwb6EYObqiSHHqdyYYdhB9g1EWyES3TmbRvh6jgX6H31-LF8ztavTy_Lh3VWUS5pxioCviykLJnXGspCF6ViXOaisIWQFVRCOypBeu609OA1ME48kaVTTPMFuj1cTY99TxBH0zWxgjY9D8MUDadCacqUypOVHaxVGGIM4M0mNJ0NO0OJ-eVoDhxN4mj2HI1MoZvj_answP1H_sDxH23jbl8</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3146912667</pqid></control><display><type>article</type><title>Leveraging large language models to improve patient education on dry eye disease</title><source>Springer Nature</source><creator>Dihan, Qais A ; Brown, Andrew D ; Chauhan, Muhammad Z ; Alzein, Ahmad F ; Abdelnaem, Seif E ; Kelso, Sean D ; Rahal, Dania A ; Park, Royce ; Ashraf, Mohammadali ; Azzam, Amr ; Morsi, Mahmoud ; Warner, David B ; Sallam, Ahmed B ; Saeed, Hajirah N ; Elhusseiny, Abdelrahman M</creator><creatorcontrib>Dihan, Qais A ; Brown, Andrew D ; Chauhan, Muhammad Z ; Alzein, Ahmad F ; Abdelnaem, Seif E ; Kelso, Sean D ; Rahal, Dania A ; Park, Royce ; Ashraf, Mohammadali ; Azzam, Amr ; Morsi, Mahmoud ; Warner, David B ; Sallam, Ahmed B ; Saeed, Hajirah N ; Elhusseiny, Abdelrahman M</creatorcontrib><description>Dry eye disease (DED) is an exceedingly common diagnosis in patients, yet recent analyses have demonstrated patient education materials (PEMs) on DED to be of low quality and readability. Our study evaluated the utility and performance of three large language models (LLMs) in enhancing and generating new patient education materials (PEMs) on dry eye disease (DED). We evaluated PEMs generated by ChatGPT-3.5, ChatGPT-4, Gemini Advanced, using three separate prompts. Prompts A and B requested they generate PEMs on DED, with Prompt B specifying a 6th-grade reading level, using the SMOG (Simple Measure of Gobbledygook) readability formula. Prompt C asked for a rewrite of existing PEMs at a 6th-grade reading level. Each PEM was assessed on readability (SMOG, FKGL: Flesch-Kincaid Grade Level), quality (PEMAT: Patient Education Materials Assessment Tool, DISCERN), and accuracy (Likert Misinformation scale). All LLM-generated PEMs in response to Prompt A and B were of high quality (median DISCERN = 4), understandable (PEMAT understandability ≥70%) and accurate (Likert Score=1). LLM-generated PEMs were not actionable (PEMAT Actionability &lt;70%). ChatGPT-4 and Gemini Advanced rewrote existing PEMs (Prompt C) from a baseline readability level (FKGL: 8.0 ± 2.4, SMOG: 7.9 ± 1.7) to targeted 6th-grade reading level; rewrites contained little to no misinformation (median Likert misinformation=1 (range: 1-2)). However, only ChatGPT-4 rewrote PEMs while maintaining high quality and reliability (median DISCERN = 4). LLMs (notably ChatGPT-4) were able to generate and rewrite PEMs on DED that were readable, accurate, and high quality. Our study underscores the value of leveraging LLMs as supplementary tools to improving PEMs.</description><identifier>ISSN: 0950-222X</identifier><identifier>ISSN: 1476-5454</identifier><identifier>EISSN: 1476-5454</identifier><identifier>DOI: 10.1038/s41433-024-03476-5</identifier><identifier>PMID: 39681711</identifier><language>eng</language><publisher>England</publisher><ispartof>Eye (London), 2024-12</ispartof><rights>2024. The Author(s), under exclusive licence to The Royal College of Ophthalmologists.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c1351-2c0efb855b2f99eb898b6235748a845cec49d15e5f3d95fef9e230f05bd6293</cites><orcidid>0000-0003-1412-375X ; 0000-0002-7385-6534</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,27922,27923</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/39681711$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Dihan, Qais A</creatorcontrib><creatorcontrib>Brown, Andrew D</creatorcontrib><creatorcontrib>Chauhan, Muhammad Z</creatorcontrib><creatorcontrib>Alzein, Ahmad F</creatorcontrib><creatorcontrib>Abdelnaem, Seif E</creatorcontrib><creatorcontrib>Kelso, Sean D</creatorcontrib><creatorcontrib>Rahal, Dania A</creatorcontrib><creatorcontrib>Park, Royce</creatorcontrib><creatorcontrib>Ashraf, Mohammadali</creatorcontrib><creatorcontrib>Azzam, Amr</creatorcontrib><creatorcontrib>Morsi, Mahmoud</creatorcontrib><creatorcontrib>Warner, David B</creatorcontrib><creatorcontrib>Sallam, Ahmed B</creatorcontrib><creatorcontrib>Saeed, Hajirah N</creatorcontrib><creatorcontrib>Elhusseiny, Abdelrahman M</creatorcontrib><title>Leveraging large language models to improve patient education on dry eye disease</title><title>Eye (London)</title><addtitle>Eye (Lond)</addtitle><description>Dry eye disease (DED) is an exceedingly common diagnosis in patients, yet recent analyses have demonstrated patient education materials (PEMs) on DED to be of low quality and readability. Our study evaluated the utility and performance of three large language models (LLMs) in enhancing and generating new patient education materials (PEMs) on dry eye disease (DED). We evaluated PEMs generated by ChatGPT-3.5, ChatGPT-4, Gemini Advanced, using three separate prompts. Prompts A and B requested they generate PEMs on DED, with Prompt B specifying a 6th-grade reading level, using the SMOG (Simple Measure of Gobbledygook) readability formula. Prompt C asked for a rewrite of existing PEMs at a 6th-grade reading level. Each PEM was assessed on readability (SMOG, FKGL: Flesch-Kincaid Grade Level), quality (PEMAT: Patient Education Materials Assessment Tool, DISCERN), and accuracy (Likert Misinformation scale). All LLM-generated PEMs in response to Prompt A and B were of high quality (median DISCERN = 4), understandable (PEMAT understandability ≥70%) and accurate (Likert Score=1). LLM-generated PEMs were not actionable (PEMAT Actionability &lt;70%). ChatGPT-4 and Gemini Advanced rewrote existing PEMs (Prompt C) from a baseline readability level (FKGL: 8.0 ± 2.4, SMOG: 7.9 ± 1.7) to targeted 6th-grade reading level; rewrites contained little to no misinformation (median Likert misinformation=1 (range: 1-2)). However, only ChatGPT-4 rewrote PEMs while maintaining high quality and reliability (median DISCERN = 4). LLMs (notably ChatGPT-4) were able to generate and rewrite PEMs on DED that were readable, accurate, and high quality. Our study underscores the value of leveraging LLMs as supplementary tools to improving PEMs.</description><issn>0950-222X</issn><issn>1476-5454</issn><issn>1476-5454</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><recordid>eNo9kE1LAzEQhoMotlb_gAfJ0ctqvndzlGJVKCjowVvIbibLyn7UZLfQf29sqzDMvDDvOwwPQteU3FHCi_soqOA8I0xkhItcZfIEzeleCClO0ZxoSTLG2OcMXcT4RUha5uQczbhWBc0pnaO3NWwh2Lrpa9zaUEPqfT3ZJLrBQRvxOOCm24RhC3hjxwb6EYObqiSHHqdyYYdhB9g1EWyES3TmbRvh6jgX6H31-LF8ztavTy_Lh3VWUS5pxioCviykLJnXGspCF6ViXOaisIWQFVRCOypBeu609OA1ME48kaVTTPMFuj1cTY99TxBH0zWxgjY9D8MUDadCacqUypOVHaxVGGIM4M0mNJ0NO0OJ-eVoDhxN4mj2HI1MoZvj_answP1H_sDxH23jbl8</recordid><startdate>20241216</startdate><enddate>20241216</enddate><creator>Dihan, Qais A</creator><creator>Brown, Andrew D</creator><creator>Chauhan, Muhammad Z</creator><creator>Alzein, Ahmad F</creator><creator>Abdelnaem, Seif E</creator><creator>Kelso, Sean D</creator><creator>Rahal, Dania A</creator><creator>Park, Royce</creator><creator>Ashraf, Mohammadali</creator><creator>Azzam, Amr</creator><creator>Morsi, Mahmoud</creator><creator>Warner, David B</creator><creator>Sallam, Ahmed B</creator><creator>Saeed, Hajirah N</creator><creator>Elhusseiny, Abdelrahman M</creator><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0003-1412-375X</orcidid><orcidid>https://orcid.org/0000-0002-7385-6534</orcidid></search><sort><creationdate>20241216</creationdate><title>Leveraging large language models to improve patient education on dry eye disease</title><author>Dihan, Qais A ; Brown, Andrew D ; Chauhan, Muhammad Z ; Alzein, Ahmad F ; Abdelnaem, Seif E ; Kelso, Sean D ; Rahal, Dania A ; Park, Royce ; Ashraf, Mohammadali ; Azzam, Amr ; Morsi, Mahmoud ; Warner, David B ; Sallam, Ahmed B ; Saeed, Hajirah N ; Elhusseiny, Abdelrahman M</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c1351-2c0efb855b2f99eb898b6235748a845cec49d15e5f3d95fef9e230f05bd6293</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Dihan, Qais A</creatorcontrib><creatorcontrib>Brown, Andrew D</creatorcontrib><creatorcontrib>Chauhan, Muhammad Z</creatorcontrib><creatorcontrib>Alzein, Ahmad F</creatorcontrib><creatorcontrib>Abdelnaem, Seif E</creatorcontrib><creatorcontrib>Kelso, Sean D</creatorcontrib><creatorcontrib>Rahal, Dania A</creatorcontrib><creatorcontrib>Park, Royce</creatorcontrib><creatorcontrib>Ashraf, Mohammadali</creatorcontrib><creatorcontrib>Azzam, Amr</creatorcontrib><creatorcontrib>Morsi, Mahmoud</creatorcontrib><creatorcontrib>Warner, David B</creatorcontrib><creatorcontrib>Sallam, Ahmed B</creatorcontrib><creatorcontrib>Saeed, Hajirah N</creatorcontrib><creatorcontrib>Elhusseiny, Abdelrahman M</creatorcontrib><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><jtitle>Eye (London)</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Dihan, Qais A</au><au>Brown, Andrew D</au><au>Chauhan, Muhammad Z</au><au>Alzein, Ahmad F</au><au>Abdelnaem, Seif E</au><au>Kelso, Sean D</au><au>Rahal, Dania A</au><au>Park, Royce</au><au>Ashraf, Mohammadali</au><au>Azzam, Amr</au><au>Morsi, Mahmoud</au><au>Warner, David B</au><au>Sallam, Ahmed B</au><au>Saeed, Hajirah N</au><au>Elhusseiny, Abdelrahman M</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Leveraging large language models to improve patient education on dry eye disease</atitle><jtitle>Eye (London)</jtitle><addtitle>Eye (Lond)</addtitle><date>2024-12-16</date><risdate>2024</risdate><issn>0950-222X</issn><issn>1476-5454</issn><eissn>1476-5454</eissn><abstract>Dry eye disease (DED) is an exceedingly common diagnosis in patients, yet recent analyses have demonstrated patient education materials (PEMs) on DED to be of low quality and readability. Our study evaluated the utility and performance of three large language models (LLMs) in enhancing and generating new patient education materials (PEMs) on dry eye disease (DED). We evaluated PEMs generated by ChatGPT-3.5, ChatGPT-4, Gemini Advanced, using three separate prompts. Prompts A and B requested they generate PEMs on DED, with Prompt B specifying a 6th-grade reading level, using the SMOG (Simple Measure of Gobbledygook) readability formula. Prompt C asked for a rewrite of existing PEMs at a 6th-grade reading level. Each PEM was assessed on readability (SMOG, FKGL: Flesch-Kincaid Grade Level), quality (PEMAT: Patient Education Materials Assessment Tool, DISCERN), and accuracy (Likert Misinformation scale). All LLM-generated PEMs in response to Prompt A and B were of high quality (median DISCERN = 4), understandable (PEMAT understandability ≥70%) and accurate (Likert Score=1). LLM-generated PEMs were not actionable (PEMAT Actionability &lt;70%). ChatGPT-4 and Gemini Advanced rewrote existing PEMs (Prompt C) from a baseline readability level (FKGL: 8.0 ± 2.4, SMOG: 7.9 ± 1.7) to targeted 6th-grade reading level; rewrites contained little to no misinformation (median Likert misinformation=1 (range: 1-2)). However, only ChatGPT-4 rewrote PEMs while maintaining high quality and reliability (median DISCERN = 4). LLMs (notably ChatGPT-4) were able to generate and rewrite PEMs on DED that were readable, accurate, and high quality. Our study underscores the value of leveraging LLMs as supplementary tools to improving PEMs.</abstract><cop>England</cop><pmid>39681711</pmid><doi>10.1038/s41433-024-03476-5</doi><orcidid>https://orcid.org/0000-0003-1412-375X</orcidid><orcidid>https://orcid.org/0000-0002-7385-6534</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 0950-222X
ispartof Eye (London), 2024-12
issn 0950-222X
1476-5454
1476-5454
language eng
recordid cdi_proquest_miscellaneous_3146912667
source Springer Nature
title Leveraging large language models to improve patient education on dry eye disease
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-10T05%3A10%3A55IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Leveraging%20large%20language%20models%20to%20improve%20patient%20education%20on%20dry%20eye%20disease&rft.jtitle=Eye%20(London)&rft.au=Dihan,%20Qais%20A&rft.date=2024-12-16&rft.issn=0950-222X&rft.eissn=1476-5454&rft_id=info:doi/10.1038/s41433-024-03476-5&rft_dat=%3Cproquest_cross%3E3146912667%3C/proquest_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c1351-2c0efb855b2f99eb898b6235748a845cec49d15e5f3d95fef9e230f05bd6293%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=3146912667&rft_id=info:pmid/39681711&rfr_iscdi=true