Tencent AI Lab Machine Translation Systems for WMT20 Chat Translation Task
Longyue Wang, Zhaopeng Tu, Xing Wang, Li Ding, Liang Ding, Shuming Shi
Abstract
This paper describes the Tencent AI Lab's submission of the WMT 2020 shared task on chat translation in EnglishGerman.Our neural machine translation (NMT) systems are built on sentence-level, document-level, nonautoregressive (NAT) and pretrained models.We integrate a number of advanced techniques into our systems, including data selection, back/forward translation, larger batch learning, model ensemble, finetuning as well as system combination.Specifically, we proposed a hybrid data selection method to select highquality and in-domain sentences from out-ofdomain data.To better capture the source contexts, we exploit to augment NAT models with evolved cross-attention.Furthermore, we explore to transfer general knowledge from four different pre-training language models to the downstream translation task.In general, we present extensive experimental results for this new translation task.Among all the participants, our GermanEnglish primary system is ranked the second in terms of BLEU scores.