Agent Attention: On the Integration of Softmax and Linear Attention
Dongchen Han, Tianzhu Ye, Yizeng Han, Zhuofan Xia, Siyuan Pan, Pengfei Wan, Shiji Song, Gao Huang
Topics & Concepts
Computer scienceSoftmax functionArtificial intelligenceArtificial neural networkNeural Networks and ApplicationsReinforcement Learning in RoboticsComputability, Logic, AI Algorithms