How Much Does Tokenization Affect Neural Machine Translation?
Miguel Domingo, Mercedes García-Martínez, Alexandre Helle, Francisco Casacuberta, Manuel García–Herranz
Topics & Concepts
Lexical analysisComputer sciencePunctuationNatural language processingMachine translationArtificial intelligenceVocabularyWord (group theory)Translation (biology)LemmatisationSpeech recognitionLinguisticsGeneMessenger RNAChemistryPhilosophyBiochemistryNatural Language Processing TechniquesTopic ModelingText Readability and Simplification