Is There One Measure-of-Fit That Fits All? A Taxonomy and Review of Measures-of-Fit for Risk-Equalization Models

• Published in: Medical Care Research and Review 2015-04, Vol.72 (2), p.220-243
• Main Authors: van Veen, S.H.C.M., van Kleef, R.C., van de Ven, W.P.M.M., van Vliet, R.C.J.A
• Format: Article
• Language: English
• Subjects: Data Interpretation, Statistical; Decision making models; Humans; Measurement errors; Measurement techniques; Models, Statistical; Performance evaluation; Policy Making; Risk Adjustment - classification; Risk Adjustment - methods; Risk assessment; Vocabularies & taxonomies
• ISSN: 1077-5587; 1552-6801
• DOI: 10.1177/1077558715572900

This study provides a taxonomy of measures-of-fit that have been used for evaluating risk-equalization models since 2000 and discusses important properties of these measures, including variations in analytic method. It is important to consider the properties of measures-of-fit and variations in analytic method, because they influence the outcomes of evaluations that eventually serve as a basis for policymaking. Analysis of 81 eligible studies resulted in the identification of 71 unique measures that were divided into 3 categories based on treatment of the prediction error: measured based on squared errors, untransformed errors, and absolute errors. We conclude that no single measure-of-fit is best across situations. The choice of a measure depends on preferences about the treatment of the prediction error and the analytic method. If the objective is measuring financial incentives for risk selection, the only adequate evaluation method is to assess the predictive performance for non-random groups.