Using narratives and numbers in performance prediction: Attitudes, confidence, and validity

• Published in: International journal of selection and assessment 2022-06, Vol.30 (2), p.216-229
• Main Authors: Niessen, A. Susan M., Kausel, Edgar E., Neumann, Marvin
• Format: Article
• Language: English
• Subjects: Accuracy; narratives; nonvalid information; performance prediction; quantification; validity
• ISSN: 0965-075X; 1468-2389
• DOI: 10.1111/ijsa.12364

In a preregistered prediction‐task experiment, we investigated the effect of narrative versus quantified information on decision‐maker perceptions, confidence, predictor weighting, and predictive accuracy when making performance predictions. We also investigated the effect of who quantifies information (the decision maker or someone else). As expected, we found higher perceived informativeness and use intentions for narrative than quantified information. Information presented narratively was also weighted somewhat more heavily than quantified information. Using quantitative information quantified by decision makers themselves yielded higher perceived autonomy and use intentions than quantitative information quantified by someone else. However, no differences in prediction confidence were found and self‐ and other‐produced quantifications received identical weight. Moreover, unexpectedly, differences in weighting did not translate to differences in predictive accuracy. Practitioner points Several authors suggested that narrative information is perceived as richer, and hence, is more influential in judgments and predictions, than quantitative information. Additionally, quantified information was suggested to be more influential when the decision‐makers quantify information themselves. We found that narrative information was perceived as more informative, yielded higher use intentions, and was weighted somewhat more heavily than quantified information. We also found higher perceived autonomy and use intentions for self‐quantified, than for other‐quantified information, but no differences in assigned weight. Notably, we found no differences in confidence or predictive accuracy. Since we did not find the expected effects on predictive accuracy, the implications for selection practice remain unclear.