Tell Us What You Really Think: A Think Aloud Protocol Analysis of the Verbal Cognitive Reflection Test

The standard interpretation of cognitive reflection tests assumes that correct responses are reflective and lured responses are unreflective. However, prior process-tracing of reflection tests has cast doubt on this interpretation. In two studies (N = 201), we deployed a validated think-aloud protoc...

Full description

Saved in:
Bibliographic Details
Published in:Journal of intelligence 2023-04, Vol.11 (4), p.76
Main Authors: Byrd, Nick, Joseph, Brianna, Gongora, Gabriela, Sirota, Miroslav
Format: Article
Language:English
Subjects:
Analysis
Cellular telephone equipment industry
Cognition & reasoning
cognitive reflection test
Data collection
Frederick, Shane
heuristics and biases
Hypotheses
judgment- and decision-making
psychometrics
think-aloud protocol analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The standard interpretation of cognitive reflection tests assumes that correct responses are reflective and lured responses are unreflective. However, prior process-tracing of reflection tests has cast doubt on this interpretation. In two studies (N = 201), we deployed a validated think-aloud protocol in-person and online to test how this assumption is satisfied by the new, validated, less familiar, and non-mathematical verbal Cognitive Reflection Test (vCRT). Verbalized thoughts in both studies revealed that most (but not all) correct responses involved reflection and that most (but not all) lured responses lacked reflection. The think-aloud protocols seemed to reflect business-as-usual performance: thinking aloud did not disrupt test performance compared to a control group. These data suggest that the vCRT usually satisfies the standard interpretation of the reflection tests (albeit not without exceptions) and that the vCRT can be a good measure of the construct theorized by the two-factor explication of 'reflection' (as deliberate and conscious).
ISSN:2079-3200
2079-3200
DOI:10.3390/jintelligence11040076