A comprehensive survey on pretrained foundation models: a history from BERT to ChatGPT

Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, Hao Peng, Jianxin Li, Jia Wu, Ziwei Liu, Pengtao Xie, Caiming Xiong, Jian Pei, Philip S. Yu, Lichao Sun

2024International Journal of Machine Learning and Cybernetics373 citationsDOI

Topics & Concepts

Computer scienceModalitiesTransformerScalabilityArtificial intelligenceMachine learningData scienceDatabaseSocial scienceQuantum mechanicsVoltageSociologyPhysicsTopic ModelingNatural Language Processing TechniquesDomain Adaptation and Few-Shot Learning