Performance of Coefficient Alpha and Its Alternatives: Effects of Different Types of Non-Normality

• Published in: Educational and psychological measurement 2023-02, Vol.83 (1), p.5-27
• Main Authors: Xiao, Leifeng, Hau, Kit-Tai
• Format: Article
• Language: English
• Subjects: Achievement Tests; Bias; Correlation; Cronbach's alpha; Evaluation Criteria; Foreign Countries; International Assessment; Likert Scales; Sample Size; Scores; Statistical Analysis; Statistical Bias; Statistical Distributions; Test Construction; Test Reliability
• ISSN: 0013-1644; 1552-3888
• DOI: 10.1177/00131644221088240

We examined the performance of coefficient alpha and its potential competitors (ordinal alpha, omega total, Revelle’s omega total [omega RT], omega hierarchical [omega h], greatest lower bound [GLB], and coefficient H) with continuous and discrete data having different types of non-normality. Results showed the estimation bias was acceptable for continuous data with varying degrees of non-normality when the scales were strong (high loadings). This bias, however, became quite large with moderate strength scales and increased with increasing non-normality. For Likert-type scales, other than omega h, most indices were acceptable with non-normal data having at least four points, and more points were better. For different exponential distributed data, omega RT and GLB were robust, whereas the bias of other indices for binomial-beta distribution was generally large. An examination of an authentic large-scale international survey suggested that its items were at worst moderately non-normal; hence, non-normality was not a big concern. We recommend (a) the demand for continuous and normally distributed data for alpha may not be necessary for less severely non-normal data; (b) for severely non-normal data, we should have at least four scale points, and more points are better; and (c) there is no single golden standard for all data types, other issues such as scale loading, model structure, or scale length are also important.