Amortized Population Gibbs Samplers with Neural Sufficient Statistics

We develop amortized population Gibbs (APG) samplers, a class of scalable methods that frames structured variational inference as adaptive importance sampling. APG samplers construct high-dimensional proposals by iterating over updates to lower-dimensional blocks of variables. We train each conditio...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2020-07
Main Authors: Wu, Hao, Zimmermann, Heiko, Sennesh, Eli, Le, Tuan Anh, Jan-Willem van de Meent
Format: Article
Language:English
Subjects:
Conjugates
Divergence
Parameterization
Population (statistical)
Probabilistic methods
Probabilistic models
Proposals
Samplers
Statistical methods
Variational methods
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We develop amortized population Gibbs (APG) samplers, a class of scalable methods that frames structured variational inference as adaptive importance sampling. APG samplers construct high-dimensional proposals by iterating over updates to lower-dimensional blocks of variables. We train each conditional proposal by minimizing the inclusive KL divergence with respect to the conditional posterior. To appropriately account for the size of the input data, we develop a new parameterization in terms of neural sufficient statistics. Experiments show that APG samplers can train highly structured deep generative models in an unsupervised manner, and achieve substantial improvements in inference accuracy relative to standard autoencoding variational methods.
ISSN:2331-8422