Two-Phase Cross-Lingual Language Model Fine-Tuning for Machine Translation Quality Estimation
Dongjun Lee
Abstract
In this paper, we describe the Bering Lab's submission to the WMT 2020 Shared Task on Quality Estimation (QE).For word-level and sentence-level translation quality estimation, we fine-tune XLM-RoBERTa, the state-of-theart cross-lingual language model, with a few additional parameters.Model training consists of two phases.We first pre-train our model on a huge artificially generated QE dataset, and then we fine-tune the model with a humanlabeled dataset.When evaluated on the WMT 2020 English-German QE test set, our systems achieve the best result on the target-side of word-level QE and the second best results on the source-side of word-level QE and sentencelevel QE among all submissions.
Topics & Concepts
Computer scienceMachine translationWord (group theory)Language modelSentenceNatural language processingTask (project management)Artificial intelligenceTranslation (biology)Quality (philosophy)Set (abstract data type)GermanTest setMathematicsLinguisticsProgramming languageEconomicsBiochemistryGeometryPhilosophyEpistemologyMessenger RNAManagementGeneChemistryNatural Language Processing TechniquesTopic ModelingText Readability and Simplification