Measuring Divergent Thinking Originality With Human Raters and Text-Mining Models: A Psychometric Comparison of Methods

Within creativity research, interest and capability in utilizing text-mining models to quantify the Originality of participant responses to Divergent Thinking tasks has risen sharply over the last decade, with many extant studies fruitfully using such methods to uncover substantive patterns among cr...

Full description

Saved in:
Bibliographic Details
Published in:Psychology of aesthetics, creativity, and the arts creativity, and the arts, 2021-11, Vol.15 (4), p.645-663
Main Authors: Dumas, Denis, Organisciak, Peter, Doherty, Michael
Format: Article
Language:English
Subjects:
Cognition & reasoning
Creativity
Data mining
Divergent Thinking
Experimentation
Female
Human
Male
Psychometrics
Quantitative psychology
Subjectivity
Test Reliability
Verbal Fluency
Websites
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Within creativity research, interest and capability in utilizing text-mining models to quantify the Originality of participant responses to Divergent Thinking tasks has risen sharply over the last decade, with many extant studies fruitfully using such methods to uncover substantive patterns among creativity-relevant constructs. However, no systematic psychometric investigation of the reliability and validity of human-rated Originality scores, and scores from various freely available text-mining systems, exists in the literature. Here we conduct such an investigation with the Alternate Uses Task. We demonstrate that, despite their inherent subjectivity, human-rated Originality scores displayed the highest reliability at both the composite and latent factor levels. However, the text-mining system GloVe 840B was highly capable of approximating human-rated scores both in its measurement properties and its correlations to various creativity-related criteria including ideational Fluency, Elaboration, Openness, Intellect, and self-reported Creative Activities. We conclude that, in conjunction with other salient indicators of creative potential, text-mining models (and especially the GloVe 840B system) are capable of supporting reliable and valid inferences about Divergent Thinking. We offer an open-access module for researchers to apply these methods to their own data via our laboratory website (https://openscoring.du.edu/).
ISSN:1931-3896
1931-390X
DOI:10.1037/aca0000319