Low-Resource Machine Translation Using Cross-Lingual Language Model Pretraining
Francis Zheng, Machel Reid, Edison Marrese-Taylor, Yutaka Matsuo
Abstract
This paper describes UTokyo's submission to the AmericasNLP 2021 Shared Task on machine translation systems for indigenous languages of the Americas. We present a lowresource machine translation system that improves translation accuracy using cross-lingual language model pretraining. Our system uses an mBART implementation of FAIRSEQ to pretrain on a large set of monolingual data from a diverse set of high-resource languages before finetuning on 10 low-resource indigenous American languages: Aymara, Bribri, Ashninka, Guaran, Wixarika, Nhuatl, Hhu, Quechua, Shipibo-Konibo, and Rarmuri. On average, our system achieved BLEU scores that were 1.64 higher and CHRF scores that were 0.0749 higher than the baseline.