Fast and Efficient Second-Order Method for Training Radial Basis Function Networks

This paper proposes an improved second order (ISO) algorithm for training radial basis function (RBF) networks. Besides the traditional parameters, including centers, widths and output weights, the input weights on the connections between input layer and hidden layer are also adjusted during the tra...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems 2012-04, Vol.23 (4), p.609-619
Main Authors: Tiantian Xie, Hao Yu, Hewlett, J., Rozycki, P., Wilamowski, B.
Format: Article
Language:English
Subjects:
Algorithm design and analysis
Algorithms
Applied sciences
Artificial intelligence
Computation
Computer science
control theory
systems
Connectionism. Neural networks
Exact sciences and technology
ISO
Jacobian matrices
Levenberg-Marquardt algorithm
Mathematical analysis
Multiplication
Networks
Neural networks
Radial basis function
Radial basis function networks
second order algorithm
Software algorithms
Studies
Training
Vectors
Vectors (mathematics)
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper proposes an improved second order (ISO) algorithm for training radial basis function (RBF) networks. Besides the traditional parameters, including centers, widths and output weights, the input weights on the connections between input layer and hidden layer are also adjusted during the training process. More accurate results can be obtained by increasing variable dimensions. Initial centers are chosen from training patterns and other parameters are generated randomly in limited range. Taking the advantages of fast convergence and powerful search ability of second order algorithms, the proposed ISO algorithm can normally reach smaller training/testing error with much less number of RBF units. During the computation process, quasi Hessian matrix and gradient vector are accumulated as the sum of related sub matrices and vectors, respectively. Only one Jacobian row is stored and used for multiplication, instead of the entire Jacobian matrix storage and multiplication. Memory reduction benefits the computation speed and allows the training of problems with basically unlimited number of patterns. Several practical discrete and continuous classification problems are applied to test the properties of the proposed ISO training algorithm.
ISSN:2162-237X
2162-2388
DOI:10.1109/TNNLS.2012.2185059