Phrase-boundary model for statistical machine translation

• Published in: Computer speech & language 2016-07, Vol.38, p.13-27
• Main Authors: Salami, Shahram, Shamsfard, Mehrnoush, Khadivi, Shahram
• Format: Article
• Language: English
• Subjects: Alignment; Boundaries; Clusters; Computer simulation; Extraction; Hierarchical models; Machine translation; Rules filtering; State of the art; Statistical machine translation; Translations
• ISSN: 0885-2308; 1095-8363
• DOI: 10.1016/j.csl.2015.11.005

•We proposed an SMT model labeling nonterminals with boundary word classes of phrases.•Word classes can be defined by POS tags and automatic word clustering.•The proposed model was filtered considering alignment pattern of phrase pairs.•Limited patterns of rules extracted from phrase pairs that are decomposable. This paper proposes a new probabilistic synchronous context-free grammar model for statistical machine translation. The model labels nonterminals with classes of boundary words on the target side of aligned phrase pairs. Labeling of the rules is performed with coarse grained and fine grained nonterminals using POS tags and word clusters trained on the target language corpus. Considering the large size of the proposed model due to the diversity of nonterminals, we have also proposed a novel approach for filtered rule extraction based on the alignment pattern of phrase pairs. Using limited patterns of rules, the extraction of hierarchical rules gets restricted from phrase pairs that are decomposable to two aligned subphrases. The proposed filtered rule extraction decreases the model size and the decoding time considerably with no significant impact on the translation quality. Using BLEU as a metric in our experiments, the proposed model achieved a notable improvement rate over the state-of-the-art hierarchical phrase-based model in the translation from Persian, French and Spanish to English language. This is applicable for all languages, even under-resourced ones having no linguistic tools.