Performance analysis of stochastic gradient algorithms under weak conditions

By using the stochastic martingale theory, convergence properties of stochastic gradient （SG） identification algorithms are studied under weak conditions. The analysis indicates that the parameter estimates by the SG algorithms consistently converge to the true parameters, as long as the information...

Full description

Saved in:
Bibliographic Details
Published in:Science China. Information sciences 2008-09, Vol.51 (9), p.1269-1280
Main Authors: Ding, Feng, Yang, HuiZhong, Liu, Fei
Format: Article
Language:English
Subjects:
Algorithms
Computer Science
Convergence
Information Systems and Communication Service
Martingales
Mathematical analysis
Parameter estimation
参数估计
弱条件
性能分析
收敛性
随机梯度算法
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:By using the stochastic martingale theory, convergence properties of stochastic gradient （SG） identification algorithms are studied under weak conditions. The analysis indicates that the parameter estimates by the SG algorithms consistently converge to the true parameters, as long as the information vector is persistently exciting （i.e., the data product moment matrix has a bounded condition number） and that the process noises are zero mean and uncorrelated. These results remove the strict assumptions, made in existing references, that the noise variances and high-order moments exist, and the processes are stationary and ergodic and the strong persis- tent excitation condition holds. This contribution greatly relaxes the convergence conditions of stochastic gradient algorithms. The simulation results with bounded and unbounded noise variances confirm the convergence conclusions proposed.
ISSN:1009-2757
1674-733X
1862-2836
1869-1919
DOI:10.1007/s11432-008-0117-y