The Effects of Questionnaire Length on the Relative Impact of Response Styles in Ambulatory Assessment

• Published in: Multivariate behavioral research 2024-09, Vol.59 (5), p.1043-1057
• Main Authors: Hasselhorn, Kilian, Ottenstein, Charlotte, Meiser, Thorsten, Lischetzke, Tanja
• Format: Article
• Language: English
• Subjects: Ambulatory assessment; multidimensional IRTree models; multilevel structural equation modeling; questionnaire length; Questionnaires; response styles
• ISSN: 0027-3171; 1532-7906; 1532-7906
• DOI: 10.1080/00273171.2024.2354233

Ambulatory assessment (AA) is becoming an increasingly popular research method in the fields of psychology and life science. Nevertheless, knowledge about the effects that design choices, such as questionnaire length (i.e., number of items per questionnaire), have on AA data quality is still surprisingly restricted. Additionally, response styles (RS), which threaten data quality, have hardly been analyzed in the context of AA. The aim of the current research was to experimentally manipulate questionnaire length and investigate the association between questionnaire length and RS in an AA study. We expected that the group with the longer (82-item) questionnaire would show greater reliance on RS relative to the substantive traits than the group with the shorter (33-item) questionnaire. Students (n = 284) received questionnaires three times a day for 14 days. We used a multigroup two-dimensional item response tree model in a multilevel structural equation modeling framework to estimate midpoint and extreme RS in our AA study. We found that the long questionnaire group showed a greater reliance on RS relative to trait-based processes than the short questionnaire group. Although further validation of our findings is necessary, we hope that researchers consider our findings when planning an AA study in the future.