Gated dynamic convolutions with deep layer fusion for abstractive document summarization

We present a novel abstractive document summarization based on the recently proposed dynamic convolutional encoder-decoder architectures. We address several aspects of summarization that are not well modeled by the basic architecture, by integrating multiple layers of the encoder, controlling inform...

Full description

Saved in:
Bibliographic Details
Published in:Computer speech & language 2021-03, Vol.66, p.101159, Article 101159
Main Authors: Kwon, Hongseok, Go, Byung-Hyun, Park, Juhong, Lee, Wonkee, Jeong, Yewon, Lee, Jong-Hyeok
Format: Article
Language:English
Subjects:
Convolutional encoder-decoder
Deep layer fusion
Document summarization
Gated dynamic convolutions
Text generation
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a novel abstractive document summarization based on the recently proposed dynamic convolutional encoder-decoder architectures. We address several aspects of summarization that are not well modeled by the basic architecture, by integrating multiple layers of the encoder, controlling information flow in the hierarchy, and exploiting external knowledge. First, we propose a simple and efficient deep layer fusion to extract salient information from the encoder layers. Second, we propose a gating mechanism to control and maintain important contextual information through the encoder-decoder layers into dynamic convolutions. Lastly, we put part-of-speech information into the model as external knowledge to better predict filters for dynamic convolutions. We evaluate our model using ROUGE metrics on three different datasets: CNN-DM, NEWSROOM-ABS, and XSUM. Experimental results show that the proposed model outperforms the state-of-the-art abstractive models on NEWSROOM-ABS and XSUM and shows comparable scores on CNN-DM.
ISSN:0885-2308
1095-8363
DOI:10.1016/j.csl.2020.101159