Bayesian neural networks for classification: how useful is the evidence framework?

This paper presents an empirical assessment of the Bayesian evidence framework for neural networks using four synthetic and four real-world classification problems. We focus on three issues; model selection, automatic relevance determination (ARD) and the use of committees. Model selection using the...

Full description

Saved in:
Bibliographic Details
Published in:Neural networks 1999-07, Vol.12 (6), p.877-892
Main Authors: Penny, W.D., Roberts, S.J.
Format: Article
Language:English
Subjects:
Applied sciences
Artificial intelligence
Bayesian
Classification
Committees
Computer science
control theory
systems
Connectionism. Neural networks
Evidence
Exact sciences and technology
Feature selection
Model selection
Multi-layer perceptron
Regularization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper presents an empirical assessment of the Bayesian evidence framework for neural networks using four synthetic and four real-world classification problems. We focus on three issues; model selection, automatic relevance determination (ARD) and the use of committees. Model selection using the evidence criterion is only tenable if the number of training examples exceeds the number of network weights by a factor of five or ten. With this number of available examples, however, cross-validation is a viable alternative. The ARD feature selection scheme is only useful in networks with many hidden units and for data sets containing many irrelevant variables. ARD is also useful as a hard feature selection method. Results on applying the evidence framework to the real-world data sets showed that committees of Bayesian networks achieved classification accuracies similar to the best alternative methods. Importantly, this was achievable with a minimum of human intervention.
ISSN:0893-6080
1879-2782
DOI:10.1016/S0893-6080(99)00040-4