Deep de Finetti: Recovering Topic Distributions from Large Language Models

Large language models (LLMs) can produce long, coherent passages of text, suggesting that LLMs, although trained on next-word prediction, must represent the latent structure that characterizes a document. Prior work has found that internal representations of LLMs encode one aspect of latent structur...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-12
Main Authors: Zhang, Liyi, McCoy, R Thomas, Sumers, Theodore R, Jian-Qiao Zhu, Griffiths, Thomas L
Format: Article
Language:English
Subjects:
Bayesian analysis
Dirichlet problem
Documents
Hypotheses
Large language models
Representations
Statistical analysis
Statistical inference
Syntax
Synthetic data
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Large language models (LLMs) can produce long, coherent passages of text, suggesting that LLMs, although trained on next-word prediction, must represent the latent structure that characterizes a document. Prior work has found that internal representations of LLMs encode one aspect of latent structure, namely syntax; here we investigate a complementary aspect, namely the document's topic structure. We motivate the hypothesis that LLMs capture topic structure by connecting LLM optimization to implicit Bayesian inference. De Finetti's theorem shows that exchangeable probability distributions can be represented as a mixture with respect to a latent generating distribution. Although text is not exchangeable at the level of syntax, exchangeability is a reasonable starting assumption for topic structure. We thus hypothesize that predicting the next token in text will lead LLMs to recover latent topic distributions. We examine this hypothesis using Latent Dirichlet Allocation (LDA), an exchangeable probabilistic topic model, as a target, and we show that the representations formed by LLMs encode both the topics used to generate synthetic data and those used to explain natural corpus data.
ISSN:2331-8422