Dealing with measurement error in list experiments: Choosing the right control list design

List experiments are widely used in the social sciences to elicit truthful responses to sensitive questions. Yet, the research design commonly suffers from the problem of measurement error in the form of non-strategic respondent error, where some inattentive participants might provide random respons...

Full description

Saved in:
Bibliographic Details
Published in:Research & politics 2021-05, Vol.8 (2)
Main Authors: Agerberg, Mattias, Tannenberg, Marcus
Format: Article
Language:English
Subjects:
Bias
control list design
Design
Experiments
List experiment
Measurement errors
non-strategic respondent error
placebo item
Political Science
Research design
Respondents
sensitive questions
Social sciences
Statsvetenskap
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:List experiments are widely used in the social sciences to elicit truthful responses to sensitive questions. Yet, the research design commonly suffers from the problem of measurement error in the form of non-strategic respondent error, where some inattentive participants might provide random responses. This type of error can result in severely biased estimates. A recently proposed solution is the use of a necessarily false placebo item to equalize the length of the treatment and control lists in order to alleviate concerns about respondent error. In this paper we show theoretically that placebo items do not in general eliminate bias caused by non-strategic respondent error. We introduce a new option, the mixed control list, and show how researchers can choose between different control list designs to minimize the problems caused by inattentive respondents. We provide researchers with practical guidance to think carefully about the bias that inattentive respondents might cause in a given application of the list experiment. We also report results from a large novel list experiment fielded to over 4900 respondents, specifically designed to illustrate our theoretical argument and recommendations.
ISSN:2053-1680
2053-1680
DOI:10.1177/20531680211013154