RECENT TRENDS IN STOCHASTIC GRADIENT DESCENT FOR MACHINE LEARNING AND BIG DATA

Stochastic Gradient Descent (SGD), also known as stochastic approximation, refers to certain simple iterative structures used for solving stochastic optimization and root finding problems. The identifying feature of SGD is that, much like in gradient descent for deterministic optimization, each succ...

Full description

Saved in:
Bibliographic Details
Main Authors: Newton, David, Pasupathy, Raghu, Yousefian, Farzad
Format: Conference Proceeding
Language:English
Subjects:
Complexity theory
Inference algorithms
Machine learning
Optimization
Portfolios
Stochastic processes
Tutorials
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Stochastic Gradient Descent (SGD), also known as stochastic approximation, refers to certain simple iterative structures used for solving stochastic optimization and root finding problems. The identifying feature of SGD is that, much like in gradient descent for deterministic optimization, each successive iterate in the recursion is determined by adding an appropriately scaled gradient estimate to the prior iterate. Owing to several factors, SGD has become the leading method to solve optimization problems arising within large-scale machine learning and "big data" contexts such as classification and regression. This tutorial covers the basics of SGD with an emphasis on modern developments. The tutorial starts with examples where SGD is applicable, and then details important flavors of SGD and reported complexity calculations.
ISSN:1558-4305
DOI:10.1109/WSC.2018.8632351