Accounting for Response Styles: Leveraging the Benefits of Combining Response Process Data Collection and Response Process Analysis Methods

• Published in: Measurement (Mahwah, N.J.) N.J.), 2022-07, Vol.20 (3), p.151-174
• Main Authors: Leventhal, Brian C, Gregg, Nikole, Ames, Allison J.
• Format: Article
• Language: English
• Subjects: Data Analysis; Data Collection; irtrees; Item Response Theory; Likert Scales; Response process models; Response Style (Tests); response styles; Test Construction; Test Interpretation; Test Items
• ISSN: 1536-6367; 1536-6359
• DOI: 10.1080/15366367.2021.1953315

Response styles introduce construct-irrelevant variance as a result of respondents systematically responding to Likert-type items regardless of content. Methods to account for response styles through data analysis as well as approaches to mitigating the effects of response styles during data collection have been well-documented. Recent approaches to modeling Likert responses, such as the IRTree model, rely on the response process individuals take when answering item responses. In this study, we advocate for the use of IRTrees to analyze Likert items in addition to using the hypothesized response process to design new items. Combining these two approaches facilitates answering Likert item design questions that have plagued researchers. These include the interpretation of a middle response option, the optimal number of response options, and how to label the response options. We present 7 research questions that could be answered using this new approach, outline methods of data collection and analysis for each, and present results from an empirical example to address one of these seven questions.