Misunderstandings and omissions in textbook accounts of effect sizes

• Published in: The British journal of psychology 2020-05, Vol.111 (2), p.395-410
• Main Author: Morris, Paul H.
• Format: Article
• Language: English
• Subjects: Behavioral sciences; effect size; methodology; Miscommunication; Null hypothesis; null hypothesis significance testing; Psychology; Reliance; Researchers; Statistical analysis; statistical reporting; Textbooks; the new statistics
• ISSN: 0007-1269; 2044-8295
• DOI: 10.1111/bjop.12401

There have been frequent attempts in psychology to reduce the reliance on null hypothesis significance testing (NHST) as the criterion for establishing the importance of results. Many authorities now recommend the reporting of effect sizes (ESs) as a supplement or alternative to NHST. However, there is extensive specialist literature highlighting problems associated with the use and interpretation of ESs. A review of the coverage of ESs in over 100 textbooks on statistical analysis in behavioural science revealed widespread neglect of ESs and the relevant critical issues that have widespread coverage in the more specialist literature. For example, many textbooks claim that ESs should be interpreted as a simple measure of the practical real‐world importance of a result despite the fact that ESs are profoundly influenced by features of design and analysis strategy. We seek to highlight areas of misunderstanding about ESs found in the pedagogical literature in the light of the more specialist literature and make recommendations to researchers for the appropriate use and interpretation of ESs. This is critical as statistics textbooks have a crucial role in the education of researchers.