Bias and the probability of generalization

In order to be useful, a learning algorithm must be able to generalize well when faced with inputs not previously presented to the system. A bias is necessary for any generalization, and as shown by several researchers in recent years, no bias can lead to strictly better generalization than any othe...

Full description

Saved in:
Bibliographic Details
Main Authors: Wilson, D.R., Martinez, T.R.
Format: Conference Proceeding
Language:English
Subjects:
Artificial intelligence
Artificial neural networks
Computer science
Laboratories
Machine learning
Machine learning algorithms
Nerve fibers
Neural networks
Nominations and elections
Table lookup
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In order to be useful, a learning algorithm must be able to generalize well when faced with inputs not previously presented to the system. A bias is necessary for any generalization, and as shown by several researchers in recent years, no bias can lead to strictly better generalization than any other when summed over all possible functions or applications. The paper provides examples to illustrate this fact, but also explains how a bias or learning algorithm can be "better" than another in practice when the probability of the occurrence of functions is taken into account. It shows how domain knowledge and an understanding of the conditions under which each learning algorithm performs well can be used to increase the probability of accurate generalization, and identifies several of the conditions that should be considered when attempting to select an appropriate bias for a particular problem.
DOI:10.1109/IIS.1997.645199