Effect of variance ratio on ANOVA robustness: Might 1.5 be the limit?

• Published in: Behavior research methods 2018-06, Vol.50 (3), p.937-962
• Main Authors: Blanca, María J., Alarcón, Rafael, Arnau, Jaume, Bono, Roser, Bendayan, Rebecca
• Format: Article
• Language: English
• Subjects: Analysis of Variance; Behavioral Science and Psychology; Cognitive Psychology; Computer Simulation; Humans; Monte Carlo Method; Psychology; Sample Size
• ISSN: 1554-3528; 1554-3528
• DOI: 10.3758/s13428-017-0918-2

Inconsistencies in the research findings on F -test robustness to variance heterogeneity could be related to the lack of a standard criterion to assess robustness or to the different measures used to quantify heterogeneity. In the present paper we use Monte Carlo simulation to systematically examine the Type I error rate of F -test under heterogeneity. One-way, balanced, and unbalanced designs with monotonic patterns of variance were considered. Variance ratio (VR) was used as a measure of heterogeneity (1.5, 1.6, 1.7, 1.8, 2, 3, 5, and 9), the coefficient of sample size variation as a measure of inequality between group sizes (0.16, 0.33, and 0.50), and the correlation between variance and group size as an indicator of the pairing between them (1, .50, 0, −.50, and −1). Overall, the results suggest that in terms of Type I error a VR above 1.5 may be established as a rule of thumb for considering a potential threat to F -test robustness under heterogeneity with unequal sample sizes.