English-Twi Parallel Corpus for Machine Translation

We present a parallel machine translation training corpus for English and Akuapem Twi of 25,421 sentence pairs. We used a transformer-based translator to generate initial translations in Akuapem Twi, which were later verified and corrected where necessary by native speakers to eliminate any occurren...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2021-04
Main Authors: Azunre, Paul, Osei, Salomey, Addo, Salomey, Lawrence Asamoah Adu-Gyamfi, Moore, Stephen, Adabankah, Bernard, Opoku, Bernard, Asare-Nyarko, Clara, Nyarko, Samuel, Amoaba, Cynthia, Appiah, Esther Dansoa, Akwerh, Felix, Richard Nii Lante Lawson, Budu, Joel, Debrah, Emmanuel, Boateng, Nana, Wisdom Ofori, Buabeng-Munkoh, Edwin, Adjei, Franklin, Isaac Kojo Essel Ampomah, Otoo, Joseph, Reindorf Borkor, Standylove Birago Mensah, Mensah, Lucien, Marcel, Mark Amoako, Anokye Acheampong Amponsah, James Ben Hayfron-Acquah
Format: Article
Language:English
Subjects:
Crowdsourcing
Datasets
Machine translation
Natural language processing
Sentences
Training
Transformers
Translations
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a parallel machine translation training corpus for English and Akuapem Twi of 25,421 sentence pairs. We used a transformer-based translator to generate initial translations in Akuapem Twi, which were later verified and corrected where necessary by native speakers to eliminate any occurrence of translationese. In addition, 697 higher quality crowd-sourced sentences are provided for use as an evaluation set for downstream Natural Language Processing (NLP) tasks. The typical use case for the larger human-verified dataset is for further training of machine translation models in Akuapem Twi. The higher quality 697 crowd-sourced dataset is recommended as a testing dataset for machine translation of English to Twi and Twi to English models. Furthermore, the Twi part of the crowd-sourced data may also be used for other tasks, such as representation learning, classification, etc. We fine-tune the transformer translation model on the training corpus and report benchmarks on the crowd-sourced test set.
ISSN:2331-8422