Sequence generative adversarial nets with a conditional discriminator

• Published in: Neurocomputing (Amsterdam) 2021-03, Vol.429, p.69-76, Article 69
• Main Authors: Yan, Yongfei, Shen, Gehui, Zhang, Song, Huang, Ting, Deng, Zhi-Hong, Yun, Unil
• Format: Article
• Language: English
• Subjects: Adversarial neural networks; Algorithm; Deep learning; Natural language processing; Text generation
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2020.10.108

The success of Generative Adversarial Networks (GANs) in image generation attracts researchers to design sequence GANs in text generation. However, the discriminators of those sequence GANs usually provide only one signal per sequence, which can not reflect detailed information, e.g. whether a token is appropriate in a sequence. In addition, maximum likelihood pre-training is typically used in those models, which is time-consuming and obscures the effects of adversarial training. To cope with these problems, we propose a new sequence GAN that consists of a conditional discriminator and a discriminator-augmented generator. The conditional discriminator provides a sequence with token-level signals. The generator is designed to approximate a discriminator-augmented distribution, which avoids pre-training. Experiments show that the conditional discriminator provides more informative guidance, and our model outperforms existing models according to metrics involving both sampling quality and sampling diversity.