When (Not) to Rely on the Reliable Change Index: A Critical Appraisal and Alternatives to Consider in Clinical Psychology

• Published in: Clinical psychology (New York, N.Y.) N.Y.), 2024-09, Vol.31 (3), p.351-366
• Main Author: McAleavey, Andrew A.
• Format: Article
• Language: English
• Subjects: Human; Index (Testing); Statistical Analysis; Statistical Estimation; Test Reliability
• ISSN: 0969-5893; 1468-2850
• DOI: 10.1037/cps0000203

The reliable change index (RCI) is a widely used statistical tool to account for measurement error when evaluating difference scores. However, there is considerable debate regarding its use. Several researchers have demonstrated ways that the RCI is insufficient or invalid, and others have defended its use for various applications. The aims of this article are to describe the formulation, rationale, and operationalization of the RCI, and critically evaluate whether it is appropriate when using self-report data, especially in clinical psychology. This evaluation finds that the RCI is rarely the best available method; is easily miscalculated, misinterpreted, and misunderstood; and produces incorrect inferences more often than alternatives, largely because it is highly insensitive to real changes. It is argued that the RCI effectively discourages the collection of appropriate data for longitudinal analysis which would benefit from more than two observations, and many applications of the RCI are inaccurate because they use inappropriate estimates of reliability. Better approaches to determining the reliability of changes are required to meet clinical needs and operationalize research questions. Several alternative methods to conceptualize and operationalize reliability of change and treatment outcomes are presented. While the RCI is easy to use, it is also easy to misuse and it fails to address the central issue: two observations of a noisy measure are insufficient data to estimate change and error. Public Health Significance Statement The reliable change index (RCI) is a statistical method used widely in psychology and medicine, yet there remains disagreement about its application. This article summarizes the concepts involved in the RCI, shows when it may be applied validly, and demonstrates its insensitivity to real changes. Alternative methods for defining the reliability of changes are discussed.