An iterative pruning algorithm for feedforward neural networks

The problem of determining the proper size of an artificial neural network is recognized to be crucial, especially for its practical implications in such important issues as learning and generalization. One popular approach for tackling this problem is commonly known as pruning and it consists of tr...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on neural networks 1997-05, Vol.8 (3), p.519-531
Main Authors: Castellano, G., Fanelli, A.M., Pelillo, M.
Format: Article
Language:English
Subjects:
Applied sciences
Artificial intelligence
Artificial neural networks
Backpropagation
Computer science
control theory
systems
Connectionism. Neural networks
Equations
Exact sciences and technology
Feedforward neural networks
Iterative algorithms
Iterative methods
Neural networks
Neurons
Pattern recognition
Testing
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The problem of determining the proper size of an artificial neural network is recognized to be crucial, especially for its practical implications in such important issues as learning and generalization. One popular approach for tackling this problem is commonly known as pruning and it consists of training a larger than necessary network and then removing unnecessary weights/nodes. In this paper, a new pruning method is developed, based on the idea of iteratively eliminating units and adjusting the remaining weights in such a way that the network performance does not worsen over the entire training set. The pruning problem is formulated in terms of solving a system of linear equations, and a very efficient conjugate gradient algorithm is used for solving it, in the least-squares sense. The algorithm also provides a simple criterion for choosing the units to be removed, which has proved to work well in practice. The results obtained over various test problems demonstrate the effectiveness of the proposed approach.
ISSN:1045-9227
1941-0093
DOI:10.1109/72.572092