Mean scores for self-report surveys completed using paper-and-pencil and computers: A meta-analytic test of equivalence

The large body of literature on the comparability of mean scores for self-report survey responses gathered using paper-and-pencil and computer data collection methodologies has yielded inconclusive results. However, no comprehensive meta-analysis has been conducted in this field, and those that are...

Full description

Saved in:
Bibliographic Details
Published in:Computers in human behavior 2018-09, Vol.86, p.153-164
Main Authors: Weigold, Arne, Weigold, Ingrid K., Natera, Sara N.
Format: Article
Language:English
Subjects:
Computer
Data acquisition
Design analysis
Equivalence
Equivalence testing
Heterogeneity
Meta-analysis
paper-and-pencil
Polls & surveys
Self report
Variance analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The large body of literature on the comparability of mean scores for self-report survey responses gathered using paper-and-pencil and computer data collection methodologies has yielded inconclusive results. However, no comprehensive meta-analysis has been conducted in this field, and those that are available for specific measures have typically not differentiated between studies using between-groups and within-subjects designs. Also, few individual studies, and no meta-analyses, have used correct statistical procedures to determine the equivalence of the two methodologies. Consequently, we conducted two meta-analyses assessing quantitative equivalence (i.e., mean scores), with the first consisting of 144 independent effect sizes from studies with between-groups designs and the second including 70 independent effect sizes from studies using within-subjects designs. Both meta-analyses assessing mean scores indicated equivalence across conditions, with large heterogeneity of variance in the between-groups analysis. Presence of others in both the paper-and-pencil and computer conditions accounted for a significant portion of this variance. Heterogeneity of variance was small for the within-subjects design analysis. Overall, results indicated that the mean scores for self-report surveys using paper-and-pencil and the computer are comparable, although heterogeneity differs for the study designs. Equivalence testing was demonstrated to be the recommended statistical procedure for this type of research.
ISSN:0747-5632
1873-7692
DOI:10.1016/j.chb.2018.04.038