Investigation Into the Effects of Using Normal Distribution Theory Methodology for Likert Scale Patient-Reported Outcome Data From Varying Underlying Distributions Including Floor/Ceiling Effects

• Published in: Value in health 2020-05, Vol.23 (5), p.625-631
• Main Authors: DeWees, Todd A., Mazza, Gina L., Golafshar, Michael A., Dueck, Amylou C.
• Format: Article
• Language: English
• Subjects: Likert scale; Normal distribution; Normality; parametric versus nonparametric tests; patient-reported outcomes; Sampling; Simulation; Statistical power; Statistics
• ISSN: 1098-3015; 1524-4733
• DOI: 10.1016/j.jval.2020.01.007

Utilization of parametric or nonparametric methods for testing Likert scale data is often debated. This 2-part simulation study aims to investigate the sampling distribution of various Likert scale distributions (including floor/ceiling effects) and analyze the effectiveness of using parametric versus nonparametric tests with varying sample sizes. We simulated populations from parametric distributions binned into Likert scales. In study 1, replicates were sampled from each distribution with sizes ranging from 5 to 150 observations, calculating means with simulated 95% CIs at each sample size. In study 2, floor/ceiling effects were introduced such that the proportion of patients responding with the lowest rating varied from approximately 40% to 90%. Two-sample tests were then conducted for the 90% floor effect distribution against all other floor distributions to determine effectiveness of parametric versus nonparametric methods via 2-sided pooled t tests and Wilcoxon rank-sum tests. Coverage of the difference in means, realized P values, relative efficiency, measures of agreement in direction, and conclusion of tests were plotted by sample size. The sampling distributions of the 1-sample means and SDs for most distributions converged quickly to Gaussian, with 95% coverage. One- and 2-sample t tests of the mean demonstrated acceptable coverage, type I error, and agreement. Simulations confirm that the sampling distribution of the mean rapidly approaches normality and appropriate tests provide adequate coverage and type I error. Two-sample t tests demonstrate appropriateness and increased statistical power gained by using parametric over nonparametric approaches, suggesting t tests should be implemented with few restrictions. •The central limit theorem has shown that under sufficient conditions or large sample sizes, the sampling distribution of the mean is Gaussian, implying the ability to use parametric testing procedures.•This article extends this premise to patient-reported outcome data to demonstrate that even with extreme distributional effects, using parametric approaches is appropriate and more powerful under most conditions, and especially with large samples.•This article allows investigators to analyze retrospectively collected Likert data with parametric approaches and to power prospective studies based on Likert data using parametric approaches, which often allows for increased interpretability of results.