Smoothing and First Order Methods: A Unified Framework

We propose a unifying framework that combines smoothing approximation with fast first order algorithms for solving nonsmooth convex minimization problems. We prove that independently of the structure of the convex nonsmooth function involved, and of the given fast first order iterative scheme, it is...

Full description

Saved in:
Bibliographic Details
Published in:SIAM journal on optimization 2012-01, Vol.22 (2), p.557-580
Main Authors: Beck, Amir, Teboulle, Marc
Format: Article
Language:English
Subjects:
Algorithms
Approximation
Convex analysis
Iterative methods
Methods
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose a unifying framework that combines smoothing approximation with fast first order algorithms for solving nonsmooth convex minimization problems. We prove that independently of the structure of the convex nonsmooth function involved, and of the given fast first order iterative scheme, it is always possible to improve the complexity rate and reach an $O(\varepsilon^{-1})$ efficiency estimate by solving an adequately smoothed approximation counterpart. Our approach relies on the combination of the notion of smoothable functions that we introduce with a natural extension of the Moreau-infimal convolution technique along with its connection to the smoothing mechanism via asymptotic functions. This allows for clarification and unification of several issues on the design, analysis, and potential applications of smoothing methods when combined with fast first order algorithms. [PUBLICATION ABSTRACT]
ISSN:1052-6234
1095-7189
DOI:10.1137/100818327