Everyday Concepts and Classification Errors: Judgments of Disability and Residence

• Published in: Journal of official statistics 2006-09, Vol.22 (3), p.385-418
• Main Authors: Tourangeau, Roger, Conrad, Frederick, Arens, Zachary, Fricker, Scott, Lee, Sunghee, Smith, Elisha
• Format: Article
• Language: English
• Subjects: Classification; Concepts; Definitions; Errors; Polls & surveys
• ISSN: 0282-423X; 2001-7367

This article examines two neglected sources of misinterpretations of survey questions. Respondents may misunderstand the questions because the survey uses an everyday term in a technical way that differs from the everyday sense (and respondents fail to recognize the difference); in addition, respondents may have trouble applying the concept to borderline cases, situations that do not map neatly onto whichever sense of the concept they are trying to use. We call these two problems misalignment of the concepts and imperfect fit between concept and instance. We examined two everyday concepts - residence and disability - that figure prominently in surveys. Our initial experiment gave respondents definitions for residence and disability and asked them to classify vignettes describing concrete instances. We constructed one definition that reflected the everyday concept (the everyday definition) and a second one that departed from it (the technical definition). The vignettes varied in how well the instance depicted matched the requirements of the two definitions (for example, some vignettes clearly met the requirements of one definition but clearly did not meet the requirements of the other). Participants who got the technical definition for residence seemed able to keep it distinct from the everyday definition but those who got the technical definition for disability seemed to fall back on their everyday concept. In addition, participants were better at classifying vignettes that closely matched the definition (central instances) than ones that did not match it so well (peripheral instances). In our second experiment, we tried to encourage participants to pay more attention to the definition of the concepts by giving the concepts unfamiliar labels (e.g., calling a residence an enumeration unit). Although the use of unfamiliar terminology did increase the proportion of respondents who consulted the definition of the concept as they judged the vignettes, there was strong evidence (especially for disability) that participants still relied on their everyday sense of the concepts. Respondents were again more accurate in classifying central than peripheral instances. People have difficulty using concepts in unfamiliar ways and, when they have to, they often make errors. Even with concepts used in their everyday senses, people have trouble classifying borderline instances, those that only partially match the definition of the concept. Category membership is graded, rat