A Survey of Model Evaluation Approaches With a Tutorial on Hierarchical Bayesian Methods

This article reviews current methods for evaluating models in the cognitive sciences, including theoretically based approaches, such as Bayes factors and minimum description length measures; simulation approaches, including model mimicry evaluations; and practical approaches, such as validation and...

Full description

Saved in:
Bibliographic Details
Published in:Cognitive science 2008-12, Vol.32 (8), p.1248-1284
Main Authors: Shiffrin, Richard M., Lee, Michael D., Kim, Woojae, Wagenmakers, Eric‐Jan
Format: Article
Language:English
Subjects:
Bayesian analysis
Bayesian model selection
Bayesian Statistics
Cognition & reasoning
Cognitive Science
Comparative Analysis
Computation
Evaluation Methods
Generalization
Hierarchical Bayesian modeling
Individual Differences
Memory
Minimum description length
Model evaluation
Model mimicry
Model selection
Prediction
Prequential analysis
Probability
Retention (Psychology)
Sciences
Simulation
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This article reviews current methods for evaluating models in the cognitive sciences, including theoretically based approaches, such as Bayes factors and minimum description length measures; simulation approaches, including model mimicry evaluations; and practical approaches, such as validation and generalization measures. This article argues that, although often useful in specific settings, most of these approaches are limited in their ability to give a general assessment of models. This article argues that hierarchical methods, generally, and hierarchical Bayesian methods, specifically, can provide a more thorough evaluation of models in the cognitive sciences. This article presents two worked examples of hierarchical Bayesian analyses to demonstrate how the approach addresses key questions of descriptive adequacy, parameter interference, prediction, and generalization in principled and coherent ways.
ISSN:0364-0213
1551-6709
DOI:10.1080/03640210802414826