AI and Veterinary Medicine: Performance of Large Language Models on the North American Licensing Examination

This study aimed to assess the performance of Large Language Models on the North American Veterinary Licensing Examination (NAVLE) and to analyze the impact of artificial intelligence in the domain of animal healthcare. For this study, a 200-question NAVLE self-assessment sourced from ICVA's we...

Full description

Saved in:
Bibliographic Details
Main Authors: Angel, Mirana, Patel, Anuj, Xing, Haiyi, Balsz, Dylan, Arbuckle, Cody, Bruyette, David, Baldi, Pierre
Format: Conference Proceeding
Language:English
Subjects:
Analytical models
Animals
Artificial Intelligence
Bard
ChatGPT
Cognition
Ethics
LLM
Medical Education
Medical services
Security
Social networking (online)
Societal Impact
Veterinary Medicine
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This study aimed to assess the performance of Large Language Models on the North American Veterinary Licensing Examination (NAVLE) and to analyze the impact of artificial intelligence in the domain of animal healthcare. For this study, a 200-question NAVLE self-assessment sourced from ICVA's website was used to evaluate the performance of three language models: GPT-3, GPT-4, and Bard. Questions involving images were omitted leaving a 164 text-only sample exam. Results were analyzed by comparing generated responses to the answer key, and scores were assigned to evaluate the models' veterinary medical reasoning capabilities. Our results showed that GPT-4 outperformed GPT-3 and Bard, passing the exam with 89 % of the text-only questions correctly. GPT-3 and Bard only achieved an accuracy of 63.4 % and 61 % respectively on the same set of questions. Language models hold promise for enhancing veterinary practices through expanded educational opportunities in the veterinary curriculum, improved diagnostic accuracy, treatment times, and efficiency. However, potential negatives include challenges in changing the current educational paradigm, reduced demand for professionals or paraprofessional concerns surrounding machine-generated decisions. Responsible and ethical integration of language models is crucial in veterinary medicine.
ISSN:2831-7343
DOI:10.1109/SNAMS60348.2023.10375414