In the Eye of the Beholder: Reply to Wilson and Shadish (2006) and Radin, Nelson, Dobyns, and Houtkooper (2006)

• Published in: Psychological bulletin 2006-07, Vol.132 (4), p.533-537
• Main Authors: Bösch, Holger, Steinkamp, Fiona, Boller, Emil
• Format: Article
• Language: English
• Subjects: Bias; Dobyns, York; Effect Size; Effect Size (Statistical); Effects; Games; Houtkooper, Joop; Human; Intention; Interrater Reliability; Meta Analysis; Misconceptions; Nelson, Roger; Numbers (Numerals); Parapsychology; Psychokinesis; Psychological aspects; Psychology; Radin, Dean; Randomness; Reader Response; Sample Size; Shadish, William R; Simulation; Social sciences; Studies; Systematic review; Test Interpretation; Wilson, David B
• ISSN: 0033-2909; 1939-1455
• DOI: 10.1037/0033-2909.132.4.533

H. Bösch, F. Steinkamp, and E. Boller's (2006) meta-analysis, which demonstrated (a) a small but highly significant overall effect, (b) a small-study effect, and (c) extreme heterogeneity, has provoked widely differing responses. After considering D. B. Wilson and W. R. Shadish's (2006) and D. Radin, R. Nelson, Y. Dobyns, and J. Houtkooper's (2006) concerns about the possible effects of psychological moderator variables, the potential for missing data, and the difficulties inherent in any meta-analytic data, the authors reaffirm their view that publication bias is the most parsimonious model to account for all 3 findings. However, until compulsory registration of trials occurs, it cannot be proven that the effect is in fact attributable to publication bias, and it remains up to the individual reader to decide how the results are best and most parsimoniously interpreted.