How does BERT capture semantics? A closer look at polysemous words
David Yenicelik, Florian Schmidt, Yannic Kilcher
Abstract
The recent paradigm shift to contextual word embeddings has seen tremendous success across a wide range of down-stream tasks. However, little is known on how the emergent relation of context and semantics manifests geometrically. We investigate polysemous words as one particularly prominent instance of semantic organization.
Topics & Concepts
Computer scienceSemantics (computer science)EmbeddingContext (archaeology)Natural language processingRelation (database)PolysemySyntaxArtificial intelligenceWord embeddingWord (group theory)Distributional semanticsLexical semanticsLinguisticsProgramming languageLexical itemDatabasePhilosophyPaleontologyBiologyTopic ModelingNatural Language Processing TechniquesSentiment Analysis and Opinion Mining