PhoBERT: Pre-trained language models for Vietnamese
Dat Quoc Nguyen, Anh Tuan Nguyen
Abstract
We present PhoBERT with two versions-PhoBERT base and PhoBERT large -the first public large-scale monolingual language models pre-trained for Vietnamese. Experimental results show that PhoBERT consistently outperforms the recent best pre-trained multilingual model XLM-R We release PhoBERT to facilitate future research and downstream applications for Vietnamese NLP. Our PhoBERT models are available at:
Topics & Concepts
VietnameseComputer scienceNatural language processingDependency (UML)Artificial intelligenceLanguage modelParsingInferenceDependency grammarNamed-entity recognitionEntity linkingNatural language understandingSequence labelingNatural languageLinguisticsKnowledge baseTask (project management)ManagementPhilosophyEconomicsTopic ModelingNatural Language Processing TechniquesMultimodal Machine Learning Applications