Virtual reality check: Statistical power, reported results, and the validity of research on the psychology of virtual reality and immersive environments

Virtual reality (VR) is a popular subject of scientific study across a variety of academic fields. In the present study we evaluate methodological trends in behavioral research on VR with respect to data collection practices, statistical reporting, and data availability. In line with this goal, we c...

Full description

Saved in:
Bibliographic Details
Published in:Computers in human behavior 2019-11, Vol.100, p.70-78
Main Authors: Lanier, Madison, Waddell, T. Franklin, Elson, Malte, Tamul, Daniel J., Ivory, James D., Przybylski, Andrew
Format: Article
Language:English
Subjects:
Data acquisition
Data analysis
meta-Science
Psychology
Quantitative methodology
Questionable research practices
Statistical power
Statistical tests
Trends
Virtual environments
Virtual reality
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Virtual reality (VR) is a popular subject of scientific study across a variety of academic fields. In the present study we evaluate methodological trends in behavioral research on VR with respect to data collection practices, statistical reporting, and data availability. In line with this goal, we conducted a meta-scientific analysis of 61 articles encompassing a total of 1122 statistical tests and highlight three emergent trends that inform our understanding of past and future studies focused on VR. Conclusions from analysis of the data include a high incidence of errors in statistical reporting, and a general lack of transparency with respect to the availability of study data. Transparency in data analysis, increased statistical power, and more careful reporting of statistical outcomes are suggested to heighten methodological rigor and improve reproducibility in the field of VR research. •We conducted a systematic review of empirical research on VR.•We included 61 articles reporting 1122 statistical tests.•We evaluated data collection practices, statistical reporting, and data availability.•There was a high error rate in statistical reporting and overall low transparency.•Paths forward to advance our empirical understanding of VR effects are discussed.
ISSN:0747-5632
1873-7692
DOI:10.1016/j.chb.2019.06.015