Towards Sequence-to-Sequence Neural Model for Croatian Abstractive Summarization

• Main Authors: Davidović, Vlatka, Ipšić, Sanda Martinčić
• Format: Conference Proceeding
• Language: English
• Subjects: Datasets; Deep learning; English language; Language; Machine translation; Multilingualism; Natural language processing; Training; Translations
• ISSN: 1847-2001; 1848-2295

Abstractive text summarization is a natural language processing task of generating a summary from an input text while preserving the meaning of the text. Today, two prevalent deep learning architectures for automatic text summarization are sequence-to-sequence and transformer. The sequence-to-sequence architecture works with sequential data where the order of words in the text is important. The models have reached state-of-the-art performance in English, but the same task is challenging for low-resourced and morphologically rich languages. Moreover, there are few publicly available datasets prepared for training the summarization task. We focus here on Croatian, since the summarization dataset for Croatian is still missing. In this paper, we propose a solution to fill this gap. The first step is creating the summarization dataset in the Croatian language, using Google machine translation from English to Croatian. With the obtained Croatian version of the dataset, we perform initial training of the sequence-to-sequence model with an attention mechanism. The preliminary results of the Croatian abstractive summarization are presented using the evaluation metrics ROUGE and BERTScore.