Positional Artefacts Propagate Through Masked Language Model Embeddings
Ziyang Luo, Artur Kulmizev, Xiaoxi Mao
Abstract
Ziyang Luo, Artur Kulmizev, Xiaoxi Mao. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Topics & Concepts
Computer scienceJoint (building)Natural language processingVolume (thermodynamics)Language modelComputational linguisticsArtificial intelligenceLinguisticsNatural languageEngineeringPhilosophyArchitectural engineeringPhysicsQuantum mechanicsTopic ModelingNatural Language Processing TechniquesSpeech and dialogue systems