What goes around comes around: an in-depth analysis of how respondents interpret ISP non-/compliance questionnaire items

• Published in: Information and computer security 2024-09, Vol.32 (4), p.459-476
• Main Authors: Gerdin, Marcus, Kolkowska, Ella, Grönlund, Åke
• Format: Article
• Language: English
• Subjects: Behavior; Compliance; Employees; Information security policy; Measuring instruments; Non-/compliance research; Noncompliance; PMT; Protection motivation theory; Questionnaires; Research methodology; Theory of planned behavior; TPB; Validation of measurement instruments; Validity; Variables
• ISSN: 2056-4961; 2056-4961; 2056-497X
• DOI: 10.1108/ICS-12-2023-0240

Purpose Research on employee non-/compliance to information security policies suffers from inconsistent results and there is an ongoing discussion about the dominating survey research methodology and its potential effect on these results. This study aims to add to this discussion by investigating discrepancies between what the authors claim to measure (theoretical properties of variables) and what they actually measure (respondents’ interpretations of the operationalized variables). This study asks: How well do respondents’ interpretations of variables correspond to their theoretical definitions? What are the characteristics of any discrepancies between variable definitions and respondent interpretations? Design/methodology/approach This study is based on in-depth interviews with 17 respondents from the Swedish public sector to understand how they interpret questionnaire measurement items operationalizing the variables Perceived Severity from Protection Motivation Theory and Attitude from Theory of Planned Behavior. Findings The authors found that respondents’ interpretations in many cases differ substantially from the theoretical definitions. Overall, the authors found four principal ways in which respondents interpreted measurement items – referred to as property contextualization, extension, alteration and oscillation – each implying more or less (dis)alignment with the intended theoretical properties of the two variables examined. Originality/value The qualitative method used proved vital to better understand respondents’ interpretations which, in turn, is key for improving self-reporting measurement instruments. To the best of the authors’ knowledge, this study is a first step toward understanding how precise and uniform definitions of variables’ theoretical properties can be operationalized into effective measurement items.