A New Perspective of Accelerated Gradient Methods: The Controlled Invariant Manifold Approach

Gradient Descent (GD) is a ubiquitous algorithm for finding the optimal solution to an optimization problem. For reduced computational complexity, the optimal solution \(\mathrm{x^*}\) of the optimization problem must be attained in a minimum number of iterations. For this objective, the paper propo...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-05
Main Authors: Gunjal, Revati, Wagh, Sushama, Nayyer, Syed Shadab, Stankovic, Alex, Singh, Navdeep M
Format: Article
Language:English
Subjects:
Algorithms
Optimization
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Gradient Descent (GD) is a ubiquitous algorithm for finding the optimal solution to an optimization problem. For reduced computational complexity, the optimal solution \(\mathrm{x^*}\) of the optimization problem must be attained in a minimum number of iterations. For this objective, the paper proposes a genesis of an accelerated gradient algorithm through the controlled dynamical system perspective. The objective of optimally reaching the optimal solution \(\mathrm{x^*}\) where \(\mathrm{\nabla f(x^*)=0}\) with a given initial condition \(\mathrm{x(0)}\) is achieved through control.
ISSN:2331-8422