Evaluating the Robustness of Parameter Estimates in Cognitive Models: A Meta-Analytic Review of Multinomial Processing Tree Models Across the Multiverse of Estimation Methods

Researchers have become increasingly aware that data-analysis decisions affect results. Here, we examine this issue systematically for multinomial processing tree (MPT) models, a popular class of cognitive models for categorical data. Specifically, we examine the robustness of MPT model parameter es...

Full description

Saved in:
Bibliographic Details
Published in:Psychological bulletin 2024-08, Vol.150 (8), p.965-1003
Main Authors: Singmann, Henrik, Heck, Daniel W., Barth, Marius, Erdfelder, Edgar, Arnold, Nina R., Aust, Frederik, Calanchini, Jimmy, Gümüsdagli, Fabian E., Horn, Sebastian S., Kellen, David, Klauer, Karl C., Matzke, Dora, Meissner, Franziska, Michalkiewicz, Martha, Schaper, Marie Luisa, Stahl, Christoph, Kuhlmann, Beatrice G., Groß, Julia
Format: Article
Language:English
Subjects:
Averages
Bayes Theorem
Bayesian analysis
Candidates
Cognition - physiology
Cognitive models
Data Interpretation, Statistical
Dissociation
Human
Humans
Magnitude Estimation
Mental Models
Models, Psychological
Models, Statistical
Moderators
Parameter estimation
Psychology
Robustness
Statistical Analysis
Statistical Estimation
Transparency
Uncertainty
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Researchers have become increasingly aware that data-analysis decisions affect results. Here, we examine this issue systematically for multinomial processing tree (MPT) models, a popular class of cognitive models for categorical data. Specifically, we examine the robustness of MPT model parameter estimates that arise from two important decisions: the level of data aggregation (complete-pooling, no-pooling, or partial-pooling) and the statistical framework (frequentist or Bayesian). These decisions span a multiverse of estimation methods. We synthesized the data from 13,956 participants (164 published data sets) with a meta-analytic strategy and analyzed the magnitude of divergence between estimation methods for the parameters of nine popular MPT models in psychology (e.g., process-dissociation, source monitoring). We further examined moderators as potential sources of divergence. We found that the absolute divergence between estimation methods was small on average (
ISSN:0033-2909
1939-1455
1939-1455
DOI:10.1037/bul0000434