Evaluation of the impact of controlled language on neural machine translation compared to other MT architectures

Many studies have shown that the application of controlled languages (CL) is an effective pre-editing technique to improve machine translation (MT) output. In this paper, we investigate whether this also holds true for neural machine translation (NMT). We compare the impact of applying nine CL rules...

Full description

Saved in:
Bibliographic Details
Published in:Machine translation 2019-06, Vol.33 (1/2), p.179-203
Main Authors: Marzouk, Shaimaa, Hansen-Schirra, Silvia
Format: Article
Language:English
Subjects:
Annotations
Artificial Intelligence
Computational Linguistics
Computer Science
Corpus analysis
Editing
English language
Error analysis
German language
Hybrid systems
Machine translation
Natural Language Processing (NLP)
Quality
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Many studies have shown that the application of controlled languages (CL) is an effective pre-editing technique to improve machine translation (MT) output. In this paper, we investigate whether this also holds true for neural machine translation (NMT). We compare the impact of applying nine CL rules on the quality of NMT output as opposed to that of rule-based, statistical, and hybrid MT by applying three methods: error annotation, human evaluation, and automatic evaluation. The analyzed data is a German corpus-based test suite of technical texts that have been translated into English by five MT systems (a neural, a rule-based, a statistical, and two hybrid MT systems). The comparison is conducted in terms of several quantitative parameters (number of errors, error types, quality ratings, and automatic evaluation metrics scores). The results show that CL rules positively affect rule-based, statistical, and hybrid MT systems. However, CL does not improve the results of the NMT system. The output of the neural system is mostly error-free both before and after CL application and has the highest quality in both scenarios among the analyzed MT systems showing a decrease in quality after applying the CL rules. The qualitative discussion of the NMT output sheds light on the problems that CL causes for this kind of MT architecture.
ISSN:0922-6567
1573-0573
DOI:10.1007/s10590-019-09233-w