Beyond the breaking point? Survey satisficing in conjoint experiments

Recent years have seen a renaissance of conjoint survey designs within social science. To date, however, researchers have lacked guidance on how many attributes they can include within conjoint profiles before survey satisficing leads to unacceptable declines in response quality. This paper addresse...

Full description

Saved in:
Bibliographic Details
Published in:Political science research and methods 2021-01, Vol.9 (1), p.53-71
Main Authors: Bansak, Kirk, Hainmueller, Jens, Hopkins, Daniel J., Yamamoto, Teppei
Format: Article
Language:English
Subjects:
Candidates
Design
Experiments
Legislatures
Original Article
Partisanship
Polls & surveys
Researchers
Respondents
Sampling
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recent years have seen a renaissance of conjoint survey designs within social science. To date, however, researchers have lacked guidance on how many attributes they can include within conjoint profiles before survey satisficing leads to unacceptable declines in response quality. This paper addresses that question using pre-registered, two-stage experiments examining choices among hypothetical candidates for US Senate or hotel rooms. In each experiment, we use the first stage to identify attributes which are perceived to be uncorrelated with the attribute of interest, so that their effects are not masked by those of the core attributes. In the second stage, we randomly assign respondents to conjoint designs with varying numbers of those filler attributes. We report the results of these experiments implemented via Amazon's Mechanical Turk and Survey Sampling International. They demonstrate that our core quantities of interest are generally stable, with relatively modest increases in survey satisficing when respondents face large numbers of attributes.
ISSN:2049-8470
2049-8489
DOI:10.1017/psrm.2019.13