How Do Large Language Models Acquire Factual Knowledge During Pretraining?
Du-Seong Chang, Hoyeon Chang, Jinho Park, Minjoon Seo, Youngkyung Seo, Sohee Yang, Seonghyeon Ye
Topics & Concepts
Computer scienceNatural language processingLinguisticsArtificial intelligenceLanguage modelNatural languageLanguage understandingFeature (linguistics)Modeling languageSemantics (computer science)Knowledge representation and reasoningPsychologyLanguage identificationCognitive scienceAction (physics)Intelligent Tutoring Systems and Adaptive LearningTopic ModelingText Readability and Simplification