A robust effect size measure Aw for MANOVA with non-normal and non-homogenous data

• Published in: Methodological innovations 2021-09, Vol.14 (3), p.1
• Main Authors: Ching-Hong Li, Johnson, Nesca, Marcello, Michael Waisman, Rory, Cheng, Yongtian, Man Chung Tze, Virginia
• Format: Article
• Language: English
• Subjects: Analysis of covariance; Gender differences; Homogeneity; Intelligence; Matrices; Methodological problems; Multivariate analysis; Normality; Psychological research; Psychology; Quantitative analysis; Simulation; Variables; Variance analysis
• ISSN: 2059-7991; 1748-0612; 2059-7991; 1748-0612
• DOI: 10.1177/20597991211055949

A common research question in psychology entails examining whether significant group differences (e.g. male and female) can be found in a list of numeric variables that measure the same underlying construct (e.g. intelligence). Researchers often use a multivariate analysis of variance (MANOVA), which is based on conventional null-hypothesis significance testing (NHST). Recently, a number of quantitative researchers have suggested reporting an effect size measure (ES) in this research scenario because of the perceived shortcomings of NHST. Thus, a number of MANOVA ESs have been proposed (e.g. generalized eta squared η Λ 2 , generalized omega squared ω Λ 2 ), but they rely on two key assumptions—multivariate normality and homogeneity of covariance matrices—which are frequently violated in psychological research. To solve this problem we propose a non-parametric (or assumptions-free) ES (Aw) for MANOVA. The new ES is developed on the basis of the non-parametric A in ANOVA. To test Aw we conducted a Monte-Carlo simulation. The results showed that Aw was accurate (robust) across different manipulated conditions—including non-normal distributions, unequal covariance matrices between groups, total sample sizes, sample size ratios, true ES values, and numbers of dependent variables—thereby providing empirical evidence supporting the use of Aw, particularly when key assumptions are violated. Implications of the proposed Aw for psychological research and other disciplines are also discussed.