Efficient Monotonic Multihead Attention

We introduce the Efficient Monotonic Multihead Attention (EMMA), a state-of-the-art simultaneous translation model with numerically-stable and unbiased monotonic alignment estimation. In addition, we present improved training and inference strategies, including simultaneous fine-tuning from an offli...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-12
Main Authors: Ma, Xutai, Sun, Anna, Ouyang, Siqi, Inaguma, Hirofumi, Paden Tomasello
Format: Article
Language:English
Subjects:
Alignment
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We introduce the Efficient Monotonic Multihead Attention (EMMA), a state-of-the-art simultaneous translation model with numerically-stable and unbiased monotonic alignment estimation. In addition, we present improved training and inference strategies, including simultaneous fine-tuning from an offline translation model and reduction of monotonic alignment variance. The experimental results demonstrate that the proposed model attains state-of-the-art performance in simultaneous speech-to-text translation on the Spanish and English translation task.
ISSN:2331-8422