Extracting Rules from Neural Networks by Pruning and Hidden-Unit Splitting

An algorithm for extracting rules from a standard three-layer feedforward neural network is proposed. The trained network is first pruned not only to remove redundant connections in the network but, more important, to detect the relevant inputs. The algorithm generates rules from the pruned network...

Full description

Saved in:
Bibliographic Details
Published in:Neural computation 1997-01, Vol.9 (1), p.205-225
Main Author: Setiono, Rudy
Format: Article
Language:English
Subjects:
Algorithms
Applied sciences
Artificial intelligence
Base Sequence
Computer science
control theory
systems
Connectionism. Neural networks
Exact sciences and technology
Exons
Introns
Learning - physiology
Neural Networks (Computer)
Neurons - physiology
Pattern recognition. Digital image processing. Computational geometry
Reproducibility of Results
Sound
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:An algorithm for extracting rules from a standard three-layer feedforward neural network is proposed. The trained network is first pruned not only to remove redundant connections in the network but, more important, to detect the relevant inputs. The algorithm generates rules from the pruned network by considering only a small number of activation values at the hidden units. If the number of inputs connected to a hidden unit is sufficiently small, then rules that describe how each of its activation values is obtained can be readily generated. Otherwise the hidden unit will be split and treated as output units, with each output unit corresponding to an activation value. A hidden layer is inserted and a new subnetwork is formed, trained, and pruned. This process is repeated until every hidden unit in the network has a relatively small number of input units connected to it. Examples on how the proposed algorithm works are shown using real-world data arising from molecular biology and signal processing. Our results show that for these complex problems, the algorithm can extract reasonably compact rule sets that have high predictive accuracy rates.
ISSN:0899-7667
1530-888X
DOI:10.1162/neco.1997.9.1.205