ArrowGAN : Learning to generate videos by learning Arrow of Time

Training GANs on videos is even more sophisticated than on images because videos have a distinguished dimension: time. While recent methods designed a dedicated architecture considering time, generated videos are still far from indistinguishable from real videos. In this paper, we introduce ArrowGAN...

Full description

Saved in:
Bibliographic Details
Published in:Neurocomputing (Amsterdam) 2021-05, Vol.438, p.223-234
Main Authors: Hong, Kibeom, Uh, Youngjung, Byun, Hyeran
Format: Article
Language:English
Subjects:
Arrow of Time
Generative Adversarial Networks
Generative model
Self-supervised learning
Video generation
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Training GANs on videos is even more sophisticated than on images because videos have a distinguished dimension: time. While recent methods designed a dedicated architecture considering time, generated videos are still far from indistinguishable from real videos. In this paper, we introduce ArrowGAN framework, where the discriminators learns to classify arrow of time as an auxiliary task and the generators tries to synthesize forward-running videos. We argue that the auxiliary task should be carefully chosen regarding the target domain. In addition, we explore categorical ArrowGAN with recent techniques in conditional image generation upon ArrowGAN framework, achieving the state-of-the-art performance on categorical video generation. Our extensive experiments validate the effectiveness of arrow of time as a self-supervisory task, and demonstrate that all our components of categorical ArrowGAN lead to the improvement regarding video inception score and Fréchet video distance on three datasets: Weizmann, UCFsports, and UCF-101.
ISSN:0925-2312
1872-8286
DOI:10.1016/j.neucom.2021.01.043