Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation

• Published in: Journal of medical Internet research 2021-11, Vol.23 (11), p.e28854
• Main Authors: Alvarado, Natasha, McVey, Lynn, Elshehaly, Mai, Greenhalgh, Joanne, Dowding, Dawn, Ruddle, Roy, Gale, Chris P, Mamas, Mamas, Doherty, Patrick, West, Robert, Feltbower, Richard, Randell, Rebecca
• Format: Article
• Language: English
• Subjects: Audits; Cardiology; Child; Clinical governance; Clinical medicine; Cognitive load; Data Collection; Delivery of Health Care; England; Feedback; Focus groups; Health care; Humans; Information technology; Intensive care; Internet; Intervention; Mortality; Original Paper; Patients; Pediatrics; Quality control; Quality Improvement; Quality management; Quality of care; Questionnaires; Scrutiny; Teaching hospitals; Usability; Usefulness
• ISSN: 1438-8871; 1439-4456; 1438-8871
• DOI: 10.2196/28854

Dashboards can support data-driven quality improvements in health care. They visualize data in ways intended to ease cognitive load and support data comprehension, but how they are best integrated into working practices needs further investigation. This paper reports the findings of a realist evaluation of a web-based quality dashboard (QualDash) developed to support the use of national audit data in quality improvement. QualDash was co-designed with data users and installed in 8 clinical services (3 pediatric intensive care units and 5 cardiology services) across 5 health care organizations (sites A-E) in England between July and December 2019. Champions were identified to support adoption. Data to evaluate QualDash were collected between July 2019 and August 2021 and consisted of 148.5 hours of observations including hospital wards and clinical governance meetings, log files that captured the extent of use of QualDash over 12 months, and a questionnaire designed to assess the dashboard's perceived usefulness and ease of use. Guided by the principles of realist evaluation, data were analyzed to understand how, why, and in what circumstances QualDash supported the use of national audit data in quality improvement. The observations revealed that variation across sites in the amount and type of resources available to support data use, alongside staff interactions with QualDash, shaped its use and impact. Sites resourced with skilled audit support staff and established reporting systems (sites A and C) continued to use existing processes to report data. A number of constraints influenced use of QualDash in these sites including that some dashboard metrics were not configured in line with user expectations and staff were not fully aware how QualDash could be used to facilitate their work. In less well-resourced services, QualDash automated parts of their reporting process, streamlining the work of audit support staff (site B), and, in some cases, highlighted issues with data completeness that the service worked to address (site E). Questionnaire responses received from 23 participants indicated that QualDash was perceived as useful and easy to use despite its variable use in practice. Web-based dashboards have the potential to support data-driven improvement, providing access to visualizations that can help users address key questions about care quality. Findings from this study point to ways in which dashboard design might be improved to optimize use and impa