Efficient Markov chain Monte Carlo inference in composite models with space alternating data augmentation

Space alternating data augmentation (SADA) was proposed by Doucet et al (2005) as a MCMC generalization of the SAGE algorithm of Fessler and Hero (1994), itself a famous variant of the EM algorithm. While SADA had previously been applied to inference in Gaussian mixture models, we show this sampler...

Full description

Saved in:
Bibliographic Details
Main Authors: Fevotte, C., Cappe, O., Cemgil, A. T.
Format: Conference Proceeding
Language:English
Subjects:
Computational modeling
Convergence
Data models
Linear regression
Markov chain Monte Carlo (MCMC)
Markov processes
Monte Carlo methods
Noise
non-negative matrix factorization (NMF)
space alternating data augmentation (SADA)
space alternating generalized expectation-maximization (SAGE)
sparse linear regression
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page 224
container_issue
container_start_page 221
container_title
container_volume
creator Fevotte, C.
Cappe, O.
Cemgil, A. T.
description Space alternating data augmentation (SADA) was proposed by Doucet et al (2005) as a MCMC generalization of the SAGE algorithm of Fessler and Hero (1994), itself a famous variant of the EM algorithm. While SADA had previously been applied to inference in Gaussian mixture models, we show this sampler to be particularly well suited for models having a composite structure, i.e., when the data may be written as a sum of latent components. The SADA sampler is shown to have favorable mixing properties and lesser storage requirement when compared to standard Gibbs sampling. We provide new alternative proofs of correctness of SADA and report results on sparse linear regression and nonnegative matrix factorization.
doi_str_mv 10.1109/SSP.2011.5967665
format conference_proceeding
fullrecord <record><control><sourceid>ieee_6IE</sourceid><recordid>TN_cdi_ieee_primary_5967665</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>5967665</ieee_id><sourcerecordid>5967665</sourcerecordid><originalsourceid>FETCH-LOGICAL-i217t-b516c03c5b46a7f097f202e57d9d06af2ef780b928b3fb723b1f1080279f77063</originalsourceid><addsrcrecordid>eNotkE1LAzEYhOMXWGvvgpf8ga1vkibZHKXUD2hRqJ5LdvdNG91Nlt2o-O8N2NMM88DADCE3DOaMgbnbbl_nHBibS6O0UvKEXLGF1BqkBn5KJlwZUQgp2RmZGV0emTKL88yEFgWUIC7JbBw_AICpkouST4hfOedrjyHRjR0-4zetD9YHuokhIV3aoY3UB4cDhhqzo3Xs-jj6DLvYYDvSH58OdOxtxrZNOASbfNjTxiZL7de-y9U5ieGaXDjbjjg76pS8P6zelk_F-uXxeXm_LjxnOhWVZKoGUctqoax2YLTjwFHqxjSgrOPodAmV4WUlXKW5qJhjeRvXxuXFSkzJ7X-vR8RdP_jODr-742viD7e3XPQ</addsrcrecordid><sourcetype>Publisher</sourcetype><iscdi>true</iscdi><recordtype>conference_proceeding</recordtype></control><display><type>conference_proceeding</type><title>Efficient Markov chain Monte Carlo inference in composite models with space alternating data augmentation</title><source>IEEE Electronic Library (IEL) Conference Proceedings</source><creator>Fevotte, C. ; Cappe, O. ; Cemgil, A. T.</creator><creatorcontrib>Fevotte, C. ; Cappe, O. ; Cemgil, A. T.</creatorcontrib><description>Space alternating data augmentation (SADA) was proposed by Doucet et al (2005) as a MCMC generalization of the SAGE algorithm of Fessler and Hero (1994), itself a famous variant of the EM algorithm. While SADA had previously been applied to inference in Gaussian mixture models, we show this sampler to be particularly well suited for models having a composite structure, i.e., when the data may be written as a sum of latent components. The SADA sampler is shown to have favorable mixing properties and lesser storage requirement when compared to standard Gibbs sampling. We provide new alternative proofs of correctness of SADA and report results on sparse linear regression and nonnegative matrix factorization.</description><identifier>ISSN: 2373-0803</identifier><identifier>ISBN: 9781457705694</identifier><identifier>ISBN: 1457705699</identifier><identifier>EISSN: 2693-3551</identifier><identifier>EISBN: 1457705702</identifier><identifier>EISBN: 9781457705687</identifier><identifier>EISBN: 9781457705700</identifier><identifier>EISBN: 1457705680</identifier><identifier>DOI: 10.1109/SSP.2011.5967665</identifier><language>eng</language><publisher>IEEE</publisher><subject>Computational modeling ; Convergence ; Data models ; Linear regression ; Markov chain Monte Carlo (MCMC) ; Markov processes ; Monte Carlo methods ; Noise ; non-negative matrix factorization (NMF) ; space alternating data augmentation (SADA) ; space alternating generalized expectation-maximization (SAGE) ; sparse linear regression</subject><ispartof>2011 IEEE Statistical Signal Processing Workshop (SSP), 2011, p.221-224</ispartof><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/5967665$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>309,310,780,784,789,790,2058,27925,54555,54920,54932</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/5967665$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Fevotte, C.</creatorcontrib><creatorcontrib>Cappe, O.</creatorcontrib><creatorcontrib>Cemgil, A. T.</creatorcontrib><title>Efficient Markov chain Monte Carlo inference in composite models with space alternating data augmentation</title><title>2011 IEEE Statistical Signal Processing Workshop (SSP)</title><addtitle>SSP</addtitle><description>Space alternating data augmentation (SADA) was proposed by Doucet et al (2005) as a MCMC generalization of the SAGE algorithm of Fessler and Hero (1994), itself a famous variant of the EM algorithm. While SADA had previously been applied to inference in Gaussian mixture models, we show this sampler to be particularly well suited for models having a composite structure, i.e., when the data may be written as a sum of latent components. The SADA sampler is shown to have favorable mixing properties and lesser storage requirement when compared to standard Gibbs sampling. We provide new alternative proofs of correctness of SADA and report results on sparse linear regression and nonnegative matrix factorization.</description><subject>Computational modeling</subject><subject>Convergence</subject><subject>Data models</subject><subject>Linear regression</subject><subject>Markov chain Monte Carlo (MCMC)</subject><subject>Markov processes</subject><subject>Monte Carlo methods</subject><subject>Noise</subject><subject>non-negative matrix factorization (NMF)</subject><subject>space alternating data augmentation (SADA)</subject><subject>space alternating generalized expectation-maximization (SAGE)</subject><subject>sparse linear regression</subject><issn>2373-0803</issn><issn>2693-3551</issn><isbn>9781457705694</isbn><isbn>1457705699</isbn><isbn>1457705702</isbn><isbn>9781457705687</isbn><isbn>9781457705700</isbn><isbn>1457705680</isbn><fulltext>true</fulltext><rsrctype>conference_proceeding</rsrctype><creationdate>2011</creationdate><recordtype>conference_proceeding</recordtype><sourceid>6IE</sourceid><recordid>eNotkE1LAzEYhOMXWGvvgpf8ga1vkibZHKXUD2hRqJ5LdvdNG91Nlt2o-O8N2NMM88DADCE3DOaMgbnbbl_nHBibS6O0UvKEXLGF1BqkBn5KJlwZUQgp2RmZGV0emTKL88yEFgWUIC7JbBw_AICpkouST4hfOedrjyHRjR0-4zetD9YHuokhIV3aoY3UB4cDhhqzo3Xs-jj6DLvYYDvSH58OdOxtxrZNOASbfNjTxiZL7de-y9U5ieGaXDjbjjg76pS8P6zelk_F-uXxeXm_LjxnOhWVZKoGUctqoax2YLTjwFHqxjSgrOPodAmV4WUlXKW5qJhjeRvXxuXFSkzJ7X-vR8RdP_jODr-742viD7e3XPQ</recordid><startdate>20110101</startdate><enddate>20110101</enddate><creator>Fevotte, C.</creator><creator>Cappe, O.</creator><creator>Cemgil, A. T.</creator><general>IEEE</general><scope>6IE</scope><scope>6IL</scope><scope>CBEJK</scope><scope>RIE</scope><scope>RIL</scope></search><sort><creationdate>20110101</creationdate><title>Efficient Markov chain Monte Carlo inference in composite models with space alternating data augmentation</title><author>Fevotte, C. ; Cappe, O. ; Cemgil, A. T.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-i217t-b516c03c5b46a7f097f202e57d9d06af2ef780b928b3fb723b1f1080279f77063</frbrgroupid><rsrctype>conference_proceedings</rsrctype><prefilter>conference_proceedings</prefilter><language>eng</language><creationdate>2011</creationdate><topic>Computational modeling</topic><topic>Convergence</topic><topic>Data models</topic><topic>Linear regression</topic><topic>Markov chain Monte Carlo (MCMC)</topic><topic>Markov processes</topic><topic>Monte Carlo methods</topic><topic>Noise</topic><topic>non-negative matrix factorization (NMF)</topic><topic>space alternating data augmentation (SADA)</topic><topic>space alternating generalized expectation-maximization (SAGE)</topic><topic>sparse linear regression</topic><toplevel>online_resources</toplevel><creatorcontrib>Fevotte, C.</creatorcontrib><creatorcontrib>Cappe, O.</creatorcontrib><creatorcontrib>Cemgil, A. T.</creatorcontrib><collection>IEEE Electronic Library (IEL) Conference Proceedings</collection><collection>IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume</collection><collection>IEEE Xplore All Conference Proceedings</collection><collection>IEEE/IET Electronic Library (IEL)</collection><collection>IEEE Proceedings Order Plans (POP All) 1998-Present</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Fevotte, C.</au><au>Cappe, O.</au><au>Cemgil, A. T.</au><format>book</format><genre>proceeding</genre><ristype>CONF</ristype><atitle>Efficient Markov chain Monte Carlo inference in composite models with space alternating data augmentation</atitle><btitle>2011 IEEE Statistical Signal Processing Workshop (SSP)</btitle><stitle>SSP</stitle><date>2011-01-01</date><risdate>2011</risdate><spage>221</spage><epage>224</epage><pages>221-224</pages><issn>2373-0803</issn><eissn>2693-3551</eissn><isbn>9781457705694</isbn><isbn>1457705699</isbn><eisbn>1457705702</eisbn><eisbn>9781457705687</eisbn><eisbn>9781457705700</eisbn><eisbn>1457705680</eisbn><abstract>Space alternating data augmentation (SADA) was proposed by Doucet et al (2005) as a MCMC generalization of the SAGE algorithm of Fessler and Hero (1994), itself a famous variant of the EM algorithm. While SADA had previously been applied to inference in Gaussian mixture models, we show this sampler to be particularly well suited for models having a composite structure, i.e., when the data may be written as a sum of latent components. The SADA sampler is shown to have favorable mixing properties and lesser storage requirement when compared to standard Gibbs sampling. We provide new alternative proofs of correctness of SADA and report results on sparse linear regression and nonnegative matrix factorization.</abstract><pub>IEEE</pub><doi>10.1109/SSP.2011.5967665</doi><tpages>4</tpages><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier ISSN: 2373-0803
ispartof 2011 IEEE Statistical Signal Processing Workshop (SSP), 2011, p.221-224
issn 2373-0803
2693-3551
language eng
recordid cdi_ieee_primary_5967665
source IEEE Electronic Library (IEL) Conference Proceedings
subjects Computational modeling
Convergence
Data models
Linear regression
Markov chain Monte Carlo (MCMC)
Markov processes
Monte Carlo methods
Noise
non-negative matrix factorization (NMF)
space alternating data augmentation (SADA)
space alternating generalized expectation-maximization (SAGE)
sparse linear regression
title Efficient Markov chain Monte Carlo inference in composite models with space alternating data augmentation
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-04T05%3A45%3A03IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-ieee_6IE&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=proceeding&rft.atitle=Efficient%20Markov%20chain%20Monte%20Carlo%20inference%20in%20composite%20models%20with%20space%20alternating%20data%20augmentation&rft.btitle=2011%20IEEE%20Statistical%20Signal%20Processing%20Workshop%20(SSP)&rft.au=Fevotte,%20C.&rft.date=2011-01-01&rft.spage=221&rft.epage=224&rft.pages=221-224&rft.issn=2373-0803&rft.eissn=2693-3551&rft.isbn=9781457705694&rft.isbn_list=1457705699&rft_id=info:doi/10.1109/SSP.2011.5967665&rft.eisbn=1457705702&rft.eisbn_list=9781457705687&rft.eisbn_list=9781457705700&rft.eisbn_list=1457705680&rft_dat=%3Cieee_6IE%3E5967665%3C/ieee_6IE%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-i217t-b516c03c5b46a7f097f202e57d9d06af2ef780b928b3fb723b1f1080279f77063%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_id=info:pmid/&rft_ieee_id=5967665&rfr_iscdi=true