Contrastive Learning with Heterogeneous Graph Attention Networks on Short Text Classification
Zhongtian Sun, Anoushka Harit, Alexandra I. Cristea, Jialin Yu, Lei Shi, Noura Al Moubayed
Abstract
Graph neural networks (GNNs) have attracted extensive interest in text classification tasks due to their expected \nsuperior performance in representation learning. However, most \nexisting studies adopted the same semi-supervised learning setting \nas the vanilla Graph Convolution Network (GCN), which requires \na large amount of labelled data during training and thus is \nless robust when dealing with large-scale graph data with fewer \nlabels. Additionally, graph structure information is normally \ncaptured by direct information aggregation via network schema \nand is highly dependent on correct adjacency information. \nTherefore, any missing adjacency knowledge may hinder the \nperformance. Addressing these problems, this paper thus proposes a novel method to learn a graph structure, NC-HGAT, \nby expanding a state-of-the-art self-supervised heterogeneous \ngraph neural network model (HGAT) with simple neighbour \ncontrastive learning. The new NC-HGAT considers the graph \nstructure information from heterogeneous graphs with multilayer perceptrons (MLPs) and delivers consistent results, despite \nthe corrupted neighbouring connections. Extensive experiments \nhave been implemented on four benchmark short-text datasets. \nThe results demonstrate that our proposed model NC-HGAT significantly outperforms state-of-the-art methods on three datasets \nand achieves competitive performance on the remaining dataset.