A robust regularized extreme learning machine for regression problems based on self-adaptive accelerated extra-gradient algorithm

The Extreme Learning Machine (ELM) technique is a machine learning approach for constructing feed-forward neural networks with a single hidden layer and their models. The ELM model can be constructed while being trained by concurrently reducing both the modeling errors and the norm of the output wei...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-01
Main Authors: Adegoke, Muideen, Jolaoso, Lateef O, Oduwole, Mardiyyah
Format: Article
Language:English
Subjects:
Adaptive algorithms
Algorithms
Artificial neural networks
Convergence
First order algorithms
Iterative methods
Machine learning
Neural networks
Robustness (mathematics)
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The Extreme Learning Machine (ELM) technique is a machine learning approach for constructing feed-forward neural networks with a single hidden layer and their models. The ELM model can be constructed while being trained by concurrently reducing both the modeling errors and the norm of the output weights. Usually, the squared loss is widely utilized in the objective function of ELM, which can be treated as a LASSO problem and hence applying the fast iterative shrinkage thresholding algorithm (FISTA). However, in this paper, the minimization problem is solved from a variational inequalities perspective giving rise to improved algorithms that are more efficient than the FISTA. A fast general extra-gradient algorithm which is a form of first-order algorithm is developed with inertial acceleration techniques and variable stepsize which is updated at every iteration. The strong convergence and linear rate of convergence of the proposed algorithm are established under mild conditions. In terms of experiments, two experiments were presented to illustrate the computational advantage of the proposed method to other algorithms.
ISSN:2331-8422