Development and Validation of a Grading System for the Quality of Cost-Effectiveness Studies

• Published in: Medical care 2003-01, Vol.41 (1), p.32-44
• Main Authors: Chiou, Chiun-Fang, Hay, Joel W., Wallace, Joel F., Bloom, Bernard S., Neumann, Peter J., Sullivan, Sean D., Yu, Hsing-Ting, Keeler, Emmett B., Henning, James M., Ofman, Joshua J.
• Format: Article
• Language: English
• Subjects: Analytical estimating; Cost-Benefit Analysis - standards; Data Interpretation, Statistical; Economic analysis; Economic research; Economic surveys; Economic systems; Economic value; Economics, Medical; Evaluation Studies as Topic; Health care economics; Health surveys; Humans; Models, Economic; Pharmaceutical preparations; Steering
• ISSN: 0025-7079; 1537-1948
• DOI: 10.1097/00005650-200301000-00007

Purpose. To provide a practical quantitative tool for appraising the quality of cost-effectiveness (CE) studies. Methods. A committee comprised of health economists selected a set of criteria for the instrument from an item pool. Data collected with a conjoint analysis survey on 120 international health economists were used to estimate weights for each criterion with a random effects regression model. To validate the grading system, a survey was sent to 60 individuals with health economics expertise. Participants first rated the quality of three CE studies on a visual analogue scale, and then evaluated each study using the grading system. Spearman rho and Wilcoxon tests were used to detect convergent validity and analysis of covariance (ANCOVA) for discriminant validity. Agreement between the global rating by experts and the grading system was also examined. Results. Sixteen criteria were selected. Their coefficient estimates ranged from 1.2 to 8.9, with a sum of 93.5 on a 100-point scale. The only insignificant criterion was "use of subgroup analyses." Both convergent validity and discriminant validity of the grading system were shown by the results of the Spearman rho (correlation coefficient = 0.78, P