An adaptive penalized maximum likelihood algorithm

The LMS algorithm is one of the most popular learning algorithms for identifying an unknown system. Many variants of the algorithm have been developed based on different problem formulations and principles. In this paper, we use the penalized maximum likelihood (PML) as a principled and unified appr...

Full description

Saved in:
Bibliographic Details
Published in:Signal processing 2008-09, Vol.88 (9), p.2323-2334
Main Authors: Deng, Guang, Ng, Wai-Yin
Format: Article
Language:English
Subjects:
Applied sciences
Exact sciences and technology
Information, signal and communications theory
Learning algorithms
LMS algorithms
Miscellaneous
Penalized maximum likelihood
Signal processing
Telecommunications and information theory
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The LMS algorithm is one of the most popular learning algorithms for identifying an unknown system. Many variants of the algorithm have been developed based on different problem formulations and principles. In this paper, we use the penalized maximum likelihood (PML) as a principled and unified approach for developing LMS-type algorithms. We study a general solution to the problem and develop algorithms to address the problems of robustness to impulsive noise and exploiting the sparseness of the system. We perform a statistical analysis of a special case of the proposed algorithm and propose a data-driven method to update the penalty parameter. We also reveal an invariant property of the algorithm. Connections with algorithms based on stochastic gradient descent are also studied. We demonstrate the competitive performance of the proposed algorithms by numerical examples and comparison with recently published algorithms.
ISSN:0165-1684
1872-7557
DOI:10.1016/j.sigpro.2008.04.001