Improving Survey Methods With Cognitive Interviews in Small- and Medium-Scale Evaluations

Findings derived from self-reported, structured survey questionnaires are commonly used in evaluation and applied research to inform policy-making and program decisions. Although there are a variety of issues related to the quality of survey evidence (e.g., sampling precision), the validity of respo...

Full description

Saved in:
Bibliographic Details
Published in:The American journal of evaluation 2012-09, Vol.33 (3), p.414-430
Main Authors: Ryan, Katherine, Gannon-Slater, Nora, Culbertson, Michael J.
Format: Article
Language:English
Subjects:
Cognition
Cognitive Development
Cognitive Processes
College Students
Decisions
Educational Policy
Evaluation Methods
Evaluation Research
Evidence
Federal Legislation
Health Needs
Interviews
National Surveys
Needs Assessment
No Child Left Behind Act 2001
Protocol Analysis
Questioning Techniques
Questionnaires
Research Design
Responses
Surveys
Test Construction
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Findings derived from self-reported, structured survey questionnaires are commonly used in evaluation and applied research to inform policy-making and program decisions. Although there are a variety of issues related to the quality of survey evidence (e.g., sampling precision), the validity of response processes–how respondents process thoughts and perceptions when answering questionnaires–is crucial. We assess the extent to which cognitive interviews, used to test survey questions as part of the process of questionnaire design and refinement, can strengthen the quality of survey evidence for small- and medium-scale evaluations. To illustrate how cognitive interviews can reveal respondents' response processes and improved questionnaire interpretations, we present excerpts from two types of evaluations (e.g., needs assessment) in two domains (e.g., education). The paper concludes with a brief summary about how the use of cognitive interviews can be augmented in survey development, refinement, and adaptation to improve survey questionnaire interpretations in evaluations with limited resources.
ISSN:1098-2140
1557-0878
DOI:10.1177/1098214012441499