LMGFuse: Language Models and Graph reasoning Fuse deeply for question answering
Aoxing Wang, Pengfei Duan, Yongbing Li, Wenyan Hu, Shengwu Xiong
Abstract
The combination of pre-trained language models (LM) and knowledge graphs (KG) can enhance the reasoning ability for Question Answering.However, previous methods typically fuse the two modalities in a shallow or knowledgedraining manner, not taking full advantage of the knowledge representation of both.How to effectively fuse the different knowledge representations is still a problem of current research.In our work, a novel model is proposed that fuses LM modal knowledge representations and graph neural network (GNN) modal knowledge representations deeply over multiple layers of modality interaction operations.Specifically, the model includes an information interaction unit, through which KG and LM knowledge can be transferred between modalities to realize knowledge fusion directly, reducing information loss.In addition, we add the context node of implicit knowledge from LM encoding in the construction of the reasoning subgraph in advance for enhancing the reasoning of the GNN.We evaluate our model on two domains in the biomedical benchmark (MedQA-USMLE) and commonsense benchmarks (OpenBookQA and CommonsenseQA).Experimental results show that our model achieves a particular improvement over existing LM and LM+KG models for reasoning over both situational constraints and structured knowledge.