Efficient Bayesian Model Selection in PARAFAC via Stochastic Thermodynamic Integration

Parallel factor analysis (PARAFAC) is one of the most popular tensor factorization models. Even though it has proven successful in diverse application fields, the performance of PARAFAC usually hinges up on the rank of the factorization, which is typically specified manually by the practitioner. In...

Full description

Saved in:
Bibliographic Details
Published in:IEEE signal processing letters 2018-05, Vol.25 (5), p.725-729
Main Authors: Nguyen, Thanh Huy, Simsekli, Umut, Richard, Gael, Cemgil, Ali Taylan
Format: Article
Language:English
Subjects:
Approximation algorithms
Bayes methods
Bayesian model selection
Computational modeling
Computer Science
Markov chain monte carlo
Mathematical model
parallel factor analysis (PARAFAC)
Signal and Image Processing
Signal processing algorithms
Stochastic processes
Tensile stress
tensor factorization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Parallel factor analysis (PARAFAC) is one of the most popular tensor factorization models. Even though it has proven successful in diverse application fields, the performance of PARAFAC usually hinges up on the rank of the factorization, which is typically specified manually by the practitioner. In this study, we develop a novel parallel and distributed Bayesian model selection technique for rank estimation in large-scale PARAFAC models. The proposed approach integrates ideas from the emerging field of stochastic gradient Markov Chain Monte Carlo, statistical physics, and distributed stochastic optimization. As opposed to the existing methods, which are based on some heuristics, our method has a clear mathematical interpretation, and has significantly lower computational requirements, thanks to data subsampling and parallelization. We provide formal theoretical analysis on the bias induced by the proposed approach. Our experiments on synthetic and large-scale real datasets show that our method is able to find the optimal model order while being significantly faster than the state-of-the-art.
ISSN:1070-9908
1558-2361
DOI:10.1109/LSP.2018.2822550