Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
Boxin Wang, Wei Ping, Peng Xu, Lawrence McAfee, Zihan Liu, Mohammad Shoeybi, Yi Dong, Oleksii Kuchaiev, Bo Li, Chaowei Xiao, Anima Anandkumar, Bryan Catanzaro
Abstract
Boxin Wang, Wei Ping, Peng Xu, Lawrence McAfee, Zihan Liu, Mohammad Shoeybi, Yi Dong, Oleksii Kuchaiev, Bo Li, Chaowei Xiao, Anima Anandkumar, Bryan Catanzaro. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.
Topics & Concepts
Ping (video games)Autoregressive modelComputer scienceNatural language processingArtificial intelligenceStatisticsMathematicsComputer securityTopic ModelingNatural Language Processing Techniques