Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora
Xisen Jin, Dejiao Zhang, Henghui Zhu, Wei Xiao, Shang-Wen Li, Xiaokai Wei, Andrew Arnold, Xiang Ren
Abstract
Xisen Jin, Dejiao Zhang, Henghui Zhu, Wei Xiao, Shang-Wen Li, Xiaokai Wei, Andrew Arnold, Xiang Ren. Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models. 2022.
Topics & Concepts
ZhàngComputer scienceNatural language processingLinguisticsLanguage modelArtificial intelligenceCognitive scienceHistoryPhilosophyPsychologyChinaArchaeologyTopic ModelingNatural Language Processing TechniquesMultimodal Machine Learning Applications