The development and validation of a meta-tool for quality appraisal of public health evidence: Meta Quality Appraisal Tool (MetaQAT)

• Published in: Public health (London) 2016-07, Vol.136, p.57-65
• Main Authors: Rosella, L, Bowman, C, Pach, B, Morgan, S, Fitzpatrick, T, Goel, V
• Format: Article
• Language: English
• Subjects: Critical appraisal; Humans; Infectious Disease; Internal Medicine; Meta-Analysis as Topic; Pilot Projects; Public Health; Public health evidence; Quality appraisal; Reproducibility of Results; Research Design - standards; Synthesis
• ISSN: 0033-3506; 1476-5616
• DOI: 10.1016/j.puhe.2015.10.027

Abstract Objectives Most quality appraisal tools were developed for clinical medicine and tend to be study-specific with a strong emphasis on risk of bias. In order to be more relevant to public health, an appropriate quality appraisal tool needs to be less reliant on the evidence hierarchy and consider practice applicability. Given the broad range of study designs used in public health, the objective of this study was to develop and validate a meta-tool that combines public health-focused principles of appraisal coupled with a set of design-specific companion tools. Study design Several design methods were used to develop and validate the tool including literature review, synthesis, and validation with a reference standard. Methods A search of critical appraisal tools relevant to public health was conducted; core concepts were collated. The resulting framework was piloted during three feedback sessions with public health practitioners. Following subsequent revisions, the final meta-tool, the Meta Quality Appraisal Tool (MetaQAT), was then validated through a content analysis of appraisals conducted by two groups of experienced public health researchers (MetaQAT vs generic appraisal form). Results The MetaQAT framework consists of four domains: relevancy, reliability, validity, and applicability. In addition, a companion tool was assembled from existing critical appraisal tools to provide study design-specific guidance on validity appraisal. Content analysis showed similar methodological and generalizability concerns were raised by both groups; however, the MetaQAT appraisers commented more extensively on applicability to public health practice. Conclusions Critical appraisal tools designed for clinical medicine have limitations for use in the context of public health. The meta-tool structure of the MetaQAT allows for rigorous appraisal, while allowing users to simultaneously appraise the multitude of study designs relevant to public health research and assess non-standard domains, such as applicability.