Asking Better Questions: How Presentation Formats Influence Information Search

• Published in: Journal of experimental psychology. Learning, memory, and cognition memory, and cognition, 2017-08, Vol.43 (8), p.1274-1297
• Main Authors: Wu, Charley M., Meder, Björn, Filimon, Flavia, Nelson, Jonathan D.
• Format: Article
• Language: English
• Subjects: Accuracy; Adolescent; Adult; Aged; Analysis of Variance; Bayes Theorem; Bayesian analysis; Bayesian Statistics; Classification; Computer System Design; Decision Making; Decision Making Skills; Educational Experiments; Experimental Design; Female; Heuristic; Heuristics; Human; Humans; Information; Information Retrieval; Information Seeking; Information Seeking Behavior; Interpersonal Influences; Judgment; Logistic Models; Male; Mathematical Concepts; Middle Aged; Online Searching; Probability; Problem Solving; Psychological Tests; Questioning Techniques; Reasoning; Search Engines; Search Strategies; Vignettes; Visual Perception; Young Adult
• ISSN: 0278-7393; 1939-1285
• DOI: 10.1037/xlm0000374

While the influence of presentation formats have been widely studied in Bayesian reasoning tasks, we present the first systematic investigation of how presentation formats influence information search decisions. Four experiments were conducted across different probabilistic environments, where subjects (N = 2,858) chose between 2 possible search queries, each with binary probabilistic outcomes, with the goal of maximizing classification accuracy. We studied 14 different numerical and visual formats for presenting information about the search environment, constructed across 6 design features that have been prominently related to improvements in Bayesian reasoning accuracy (natural frequencies, posteriors, complement, spatial extent, countability, and part-to-whole information). The posterior variants of the icon array and bar graph formats led to the highest proportion of correct responses, and were substantially better than the standard probability format. Results suggest that presenting information in terms of posterior probabilities and visualizing natural frequencies using spatial extent (a perceptual feature) were especially helpful in guiding search decisions, although environments with a mixture of probabilistic and certain outcomes were challenging across all formats. Subjects who made more accurate probability judgments did not perform better on the search task, suggesting that simple decision heuristics may be used to make search decisions without explicitly applying Bayesian inference to compute probabilities. We propose a new take-the-difference (TTD) heuristic that identifies the accuracy-maximizing query without explicit computation of posterior probabilities.