Bayesian model evidence as a practical alternative to deviance information criterion

• Published in: Royal Society open science 2018-03, Vol.5 (3), p.171519-171519
• Main Authors: Pooley, C. M., Marion, G.
• Format: Article
• Language: English
• Subjects: Bayes' Factor; Bayesian Model Evidence; Deviance Information Criterion; Marginal Likelihood; Markov Chain Monte Carlo; Mathematics; Thermodynamic Integration
• ISSN: 2054-5703; 2054-5703
• DOI: 10.1098/rsos.171519

While model evidence is considered by Bayesian statisticians as a gold standard for model selection (the ratio in model evidence between two models giving the Bayes factor), its calculation is often viewed as too computationally demanding for many applications. By contrast, the widely used deviance information criterion (DIC), a different measure that balances model accuracy against complexity, is commonly considered a much faster alternative. However, recent advances in computational tools for efficient multi-temperature Markov chain Monte Carlo algorithms, such as steppingstone sampling (SS) and thermodynamic integration schemes, enable efficient calculation of the Bayesian model evidence. This paper compares both the capability (i.e. ability to select the true model) and speed (i.e. CPU time to achieve a given accuracy) of DIC with model evidence calculated using SS. Three important model classes are considered: linear regression models, mixed models and compartmental models widely used in epidemiology. While DIC was found to correctly identify the true model when applied to linear regression models, it led to incorrect model choice in the other two cases. On the other hand, model evidence led to correct model choice in all cases considered. Importantly, and perhaps surprisingly, DIC and model evidence were found to run at similar computational speeds, a result reinforced by analytically derived expressions.