SITD-NMT: Synchronous Inference NMT with Turing Re-Translation Detection

• Main Authors: Wu, Nier, Zhuang, Xufei, Xing, Hao, Ji, Yatu, Liu, Yang, Ren, Qingdaoerji, Shi, Bao
• Format: Conference Proceeding
• Language: English
• Subjects: Bidirectional control; Neural machine translation; Neural networks; Neural Turing Machine; Predictive models; Read-Write; Synchronous Inference; Training; Transformers; Turing machines
• ISSN: 2161-4407
• DOI: 10.1109/IJCNN60899.2024.10650580

Conventional Neural Machine Translation (NMT) relies on previous tokens and the hidden state of the target for the inference of the target tokens in the decoding phase, and this left-to-right decoding approach loses the context of the target sequence from right-to-left. In addition, the vanilla attention mechanism lacks interactivity in the bilingual training phase, where the computation of the attention weights is independent at each step, which leads the decoder to disregard whether or not the current token has already been translated. In this paper, we proposed a novel interactive synchronized bi-directional inference method that uses past and future contexts to synchronously predict target sequences and incorporates Neural Turing Machine (NTM) ideas to detect historical attentional information, which relies on a read − write mechanism to update the source hidden state. We evaluate the proposed model in the WMT14 German-English translation task and the LDC Chinese-English translation task, and the experimental results show that our method improves 2.18 and 4.07 BLEU scores over the Transformer, respectively, which fully demonstrates the effectiveness of the proposed method.