Detecting and Mitigating the Ungrounded Hallucinations in Text Generation by LLMs
Zizhong Wei, Dongsheng Guo, Dengrong Huang, Qilai Zhang, Sijia Zhang, Kai Jiang, Rui Li
Abstract
Large language models (LLMs) have achieved impressive success in generating fluent and coherent texts in natural language. However, the presence of inaccurate or low-quality data can unintentionally lead to the retention of incorrect knowledge, resulting in hallucinations that hinder progress in content generation. In this paper, we propose a comprehensive framework aimed at detecting and mitigating these hallucinations. Our approach uses Named Entity Recognition (NER) and Entity Relationship (ER) models to identify hallucination entities and sentences during the detection phase. Furthermore, by incorporating prompt engineering, we effectively correct these hallucination sentences using LLM in the mitigation phase. Tests on real articles confirm the effectiveness of our approach in rectifying LLM-associated hallucinations without adding new ones, thereby enhancing their reliability and credibility.