QGFN: Controllable Greediness with Action Values

Generative Flow Networks (GFlowNets; GFNs) are a family of energy-based generative methods for combinatorial objects, capable of generating diverse and high-utility samples. However, consistently biasing GFNs towards producing high-utility samples is non-trivial. In this work, we leverage connection...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-11
Main Authors: Lau, Elaine, Lu, Stephen Zhewen, Pan, Ling, Precup, Doina, Bengio, Emmanuel
Format: Article
Language:English
Subjects:
Combinatorial analysis
Controllability
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Generative Flow Networks (GFlowNets; GFNs) are a family of energy-based generative methods for combinatorial objects, capable of generating diverse and high-utility samples. However, consistently biasing GFNs towards producing high-utility samples is non-trivial. In this work, we leverage connections between GFNs and reinforcement learning (RL) and propose to combine the GFN policy with an action-value estimate, \(Q\), to create greedier sampling policies which can be controlled by a mixing parameter. We show that several variants of the proposed method, QGFN, are able to improve on the number of high-reward samples generated in a variety of tasks without sacrificing diversity.
ISSN:2331-8422