Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference
Bang An, Jie Lyu, Zhenyi Wang, Chunyuan Li, Changwei Hu, Fei Tan, Ruiyi Zhang, Yifan Hu, Changyou Chen
Abstract
Bang An, Jie Lyu, Zhenyi Wang, Chunyuan Li, Changwei Hu, Fei Tan, Ruiyi Zhang, Yifan Hu, Changyou Chen. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.
Topics & Concepts
ZhàngComputer scienceBayesian probabilityArtificial intelligenceBayesian inferenceInferenceChenNatural language processingCognitive scienceMachine learningPsychologyHistoryArchaeologyChinaPaleontologyBiologyTopic ModelingDomain Adaptation and Few-Shot LearningMultimodal Machine Learning Applications