Online Nonnegative Matrix Factorization With Robust Stochastic Approximation

Nonnegative matrix factorization (NMF) has become a popular dimension-reduction method and has been widely applied to image processing and pattern recognition problems. However, conventional NMF learning methods require the entire dataset to reside in the memory and thus cannot be applied to large-s...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems 2012-07, Vol.23 (7), p.1087-1099
Main Authors: Guan, Naiyang, Tao, Dacheng, Luo, Zhigang, Yuan, Bo
Format: Article
Language:English
Subjects:
Algorithm design and analysis
Algorithmics. Computability. Computer arithmetics
Algorithms
Applied sciences
Approximation algorithms
Approximation methods
Artificial intelligence
Complexity theory
Computer science
control theory
systems
Computer systems and distributed systems. User interface
Convergence
Exact sciences and technology
Factorization
Learning
Mathematical analysis
Nonnegative matrix factorization (NMF)
On-line systems
Online
online NMF (ONMF)
Pattern recognition. Digital image processing. Computational geometry
robust stochastic approximation
Robustness
Software
Stochasticity
Streaming media
Studies
Theoretical computing
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Nonnegative matrix factorization (NMF) has become a popular dimension-reduction method and has been widely applied to image processing and pattern recognition problems. However, conventional NMF learning methods require the entire dataset to reside in the memory and thus cannot be applied to large-scale or streaming datasets. In this paper, we propose an efficient online RSA-NMF algorithm (OR-NMF) that learns NMF in an incremental fashion and thus solves this problem. In particular, OR-NMF receives one sample or a chunk of samples per step and updates the bases via robust stochastic approximation. Benefitting from the smartly chosen learning rate and averaging technique, OR-NMF converges at the rate of in each update of the bases. Furthermore, we prove that OR-NMF almost surely converges to a local optimal solution by using the quasi-martingale. By using a buffering strategy, we keep both the time and space complexities of one step of the OR-NMF constant and make OR-NMF suitable for large-scale or streaming datasets. Preliminary experimental results on real-world datasets show that OR-NMF outperforms the existing online NMF (ONMF) algorithms in terms of efficiency. Experimental results of face recognition and image annotation on public datasets confirm the effectiveness of OR-NMF compared with the existing ONMF algorithms.
ISSN:2162-237X
2162-2388
DOI:10.1109/TNNLS.2012.2197827