The cascade-correlation learning: a projection pursuit learning perspective

Cascade-correlation (Cascor) is a popular supervised learning architecture that dynamically grows layers of hidden neurons of fixed nonlinear activations (e.g., sigmoids), so that the network topology (size, depth) can be efficiently determined. Similar to a cascade-correlation learning network (CCL...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on neural networks 1996-03, Vol.7 (2), p.278-289
Main Authors: HWANG, J.-N, YOU, S.-S, LAY, S.-R, JOU, I.-C
Format: Article
Language:English
Subjects:
Analytical models
Applied sciences
Artificial intelligence
Backpropagation algorithms
Biological neural networks
Computer science
control theory
systems
Connectionism. Neural networks
Exact sciences and technology
Laboratories
Multi-layer neural network
Multilayer perceptrons
Network topology
Neural networks
Neurons
Supervised learning
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Cascade-correlation (Cascor) is a popular supervised learning architecture that dynamically grows layers of hidden neurons of fixed nonlinear activations (e.g., sigmoids), so that the network topology (size, depth) can be efficiently determined. Similar to a cascade-correlation learning network (CCLN), a projection pursuit learning network (PPLN) also dynamically grows the hidden neurons. Unlike a CCLN where cascaded connections from the existing hidden units to the new candidate hidden unit are required to establish high-order nonlinearity in approximating the residual error, a PPLN approximates the high-order nonlinearity by using trainable parametric or semi-parametric nonlinear smooth activations based on minimum mean squared error criterion. An analysis is provided to show that the maximum correlation training criterion used in a CCLN tends to produce hidden units that saturate and thus makes it more suitable for classification tasks instead of regression tasks as evidenced in the simulation results. It is also observed that this critical weakness in CCLN can also potentially carry over to classification tasks, such as the two-spiral benchmark used in the original CCLN paper.
ISSN:1045-9227
1941-0093
DOI:10.1109/72.485631