Fusing Label Embedding into BERT: An Efficient Improvement for Text Classification
Yijin Xiong, Yukun Feng, Hao Wu, Hidetaka Kamigaito, Manabu Okumura
Abstract
With pre-trained models, such as BERT, gaining more and more attention, plenty of research has been done to further promote their capabilities, from enhancing the experimental procedures In this paper, we propose a concise method for improving BERT's performance in text classification by utilizing a label embedding technique while keeping almost the same computational cost. Experimental results on six text classification benchmark datasets demonstrate its effectiveness.
Topics & Concepts
Computer scienceEmbeddingMulti-label classificationArtificial intelligenceInformation retrievalPattern recognition (psychology)Natural language processingText and Document Classification TechnologiesTopic ModelingNatural Language Processing Techniques