Measurement of Utility in User Access of COVID-19 Literature via AI-powered Chatbot

The advent of COVID-19 has resulted in a data deluge in the medical literature archives and there is a need for publication analytics services to augment the clinical workflow of medical users (e.g., clinicians, researchers, medical students) for literature search. Publication analytics services suc...

Full description

Saved in:
Bibliographic Details
Main Authors: Oruche, Roland, Milman, Eric D., Cheng, Xiyao, Joish, Megha, Kulkarni, Chaitra, Sharma, Agrim, Kee, Kerk, Regunath, Hariharan, Calyam, Prasad
Format: Conference Proceeding
Language:English
Subjects:
Chatbots
COVID-19
Diffusion of Innovations
Internet
Logic gates
Publication Analytics
Software measurement
Statistical Inferencing
Statistical Utility Mea-surement
Text mining
User interfaces
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The advent of COVID-19 has resulted in a data deluge in the medical literature archives and there is a need for publication analytics services to augment the clinical workflow of medical users (e.g., clinicians, researchers, medical students) for literature search. Publication analytics services such as KnowCOVID-19 science gateway with a chatbot interface viz., Vidura Advisor have been developed to mine relevant literature from archives such as e.g., CORD-19 open-source dataset. In this paper, we present a novel utility measurement framework that uses a statistical technique for z-score calculation to measure the utility in user access of COVID-19 literature with publi-cation analytics services. Our framework approach builds on a usability study of the KnowCOVID-19 science gateway to identify challenges of user adoption of publication analytics at the individual level through the assessment of user performance and perception of factors such as user interface design, functionality, and derived information insights. In addition, we detail the soft-ware features (e.g., domain-specific topic filtering, data reports in terms of drugs/genes) within KnowCOVID-19 that support our measurement framework assessments. We evaluate our proposed framework through experiments on user performance and perception by comparing KnowCOVID-19 assisted by Vidura Advisor with a standard search engine i.e., Google Scholar over a set of clinical literature search tasks. The results from our usability study for assessing user performance using a single usability metric (z-score) show a 47% higher score in application utility with KnowCOVID-19 compared to Google Scholar.
ISSN:2332-5615
DOI:10.1109/AIPR52630.2021.9762160