Improved Computation for Levenberg-Marquardt Training

The improved computation presented in this paper is aimed to optimize the neural networks learning process using Levenberg-Marquardt (LM) algorithm. Quasi-Hessian matrix and gradient vector are computed directly, without Jacobian matrix multiplication and storage. The memory limitation problem for L...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems 2010-06, Vol.21 (6), p.930-937
Main Authors: Wilamowski, Bogdan M, Hao Yu
Format: Article
Language:English
Subjects:
Algorithmics. Computability. Computer arithmetics
Algorithms
Applied sciences
Arrays
Artificial intelligence
Backpropagation algorithms
Computation
Computational efficiency
Computer networks
Computer science
control theory
systems
Computer Simulation
Connectionism. Neural networks
Control systems
Electrical equipment industry
Exact sciences and technology
Humans
Industrial control
Industrial training
Jacobian matrices
Jacobian matrix
Learning
Levenberg-Marquardt (LM) algorithm
Mathematical analysis
neural network training
Neural networks
Neural Networks (Computer)
Neurons
Nonlinear control systems
Pattern Recognition, Automated
Theoretical computing
Time Factors
Training
Vectors (mathematics)
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The improved computation presented in this paper is aimed to optimize the neural networks learning process using Levenberg-Marquardt (LM) algorithm. Quasi-Hessian matrix and gradient vector are computed directly, without Jacobian matrix multiplication and storage. The memory limitation problem for LM training is solved. Considering the symmetry of quasi-Hessian matrix, only elements in its upper/lower triangular array need to be calculated. Therefore, training speed is improved significantly, not only because of the smaller array stored in memory, but also the reduced operations in quasi-Hessian matrix calculation. The improved memory and time efficiencies are especially true for large sized patterns training.
ISSN:1045-9227
2162-237X
1941-0093
2162-2388
DOI:10.1109/TNN.2010.2045657