Assessing the impact of healthcare research: A systematic review of methodological frameworks

• Published in: PLoS medicine 2017-08, Vol.14 (8), p.e1002370-e1002370
• Main Authors: Cruz Rivera, Samantha, Kyte, Derek G, Aiyegbusi, Olalekan Lee, Keeley, Thomas J, Calvert, Melanie J
• Format: Article
• Language: English
• Subjects: Acquired immune deficiency syndrome; AIDS; Consortia; Databases, Factual - statistics & numerical data; Economic impact; Funding; Health care; Health services; Health Services Research - methods; Health Services Research - standards; Humans; Influence; Information services; Managed competition; Measurement; Medical research; Medicine and Health Sciences; Nursing; On-line systems; Primary care; Research and Analysis Methods; Researchers; Resource allocation; Science Policy; Social Sciences; Society; Studies; Supervision
• ISSN: 1549-1676; 1549-1277; 1549-1676
• DOI: 10.1371/journal.pmed.1002370

Increasingly, researchers need to demonstrate the impact of their research to their sponsors, funders, and fellow academics. However, the most appropriate way of measuring the impact of healthcare research is subject to debate. We aimed to identify the existing methodological frameworks used to measure healthcare research impact and to summarise the common themes and metrics in an impact matrix. Two independent investigators systematically searched the Medical Literature Analysis and Retrieval System Online (MEDLINE), the Excerpta Medica Database (EMBASE), the Cumulative Index to Nursing and Allied Health Literature (CINAHL+), the Health Management Information Consortium, and the Journal of Research Evaluation from inception until May 2017 for publications that presented a methodological framework for research impact. We then summarised the common concepts and themes across methodological frameworks and identified the metrics used to evaluate differing forms of impact. Twenty-four unique methodological frameworks were identified, addressing 5 broad categories of impact: (1) 'primary research-related impact', (2) 'influence on policy making', (3) 'health and health systems impact', (4) 'health-related and societal impact', and (5) 'broader economic impact'. These categories were subdivided into 16 common impact subgroups. Authors of the included publications proposed 80 different metrics aimed at measuring impact in these areas. The main limitation of the study was the potential exclusion of relevant articles, as a consequence of the poor indexing of the databases searched. The measurement of research impact is an essential exercise to help direct the allocation of limited research resources, to maximise research benefit, and to help minimise research waste. This review provides a collective summary of existing methodological frameworks for research impact, which funders may use to inform the measurement of research impact and researchers may use to inform study design decisions aimed at maximising the short-, medium-, and long-term impact of their research.