An advanced active set L-BFGS algorithm for training weight-constrained neural networks

In this work, a new advanced active set limited memory BFGS (Broyden–Fletcher–Goldfarb–Shanno) algorithm is proposed for efficiently training weight-constrained neural networks, called AA-L-BFGS. The proposed algorithm possesses the significant property of approximating the curvature of the error fu...

Full description

Saved in:
Bibliographic Details
Published in:Neural computing & applications 2020-06, Vol.32 (11), p.6669-6684
Main Author: Livieris, Ioannis E.
Format: Article
Language:English
Subjects:
Algorithms
Artificial Intelligence
Computational Biology/Bioinformatics
Computational Science and Engineering
Computer Science
Convergence
Data Mining and Knowledge Discovery
Error functions
Image Processing and Computer Vision
Neural networks
Probability and Statistics in Computer Science
S.I. : Brain inspired Computing&Machine Learning Applied Research-BISMLARE
Training
Weight
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this work, a new advanced active set limited memory BFGS (Broyden–Fletcher–Goldfarb–Shanno) algorithm is proposed for efficiently training weight-constrained neural networks, called AA-L-BFGS. The proposed algorithm possesses the significant property of approximating the curvature of the error function with high-order accuracy by utilizing the theoretically advanced secant condition proposed by Livieris and Pintelas (Appl Math Comput 221:491–502, 2013). Moreover, the global convergence of the proposed algorithm is established provided that the line search satisfies the modified Armijo condition. The presented numerical experiments illustrate the efficiency of the proposed AA-L-BFGS, providing empirical evidence that it significantly accelerates the convergence of the training process.
ISSN:0941-0643
1433-3058
DOI:10.1007/s00521-019-04689-6