A zero-shot self-improving NER method for cyber threat intelligence via knowledge injection
Yingchang Jiang, Hao Hu, Yunpeng Li, Feiyang Li, Changzhi Zhao, Cheng Chen, Yuling Liu
Abstract
Abstract The rapid evolution of cyber threats demands efficient entity extraction from Cyber Threat Intelligence (CTI) reports to support proactive analysis and sharing. Current methods for CTI extraction falter due to a lack of domain knowledge, which can lead to the overlooking of critical entities. Moreover, the hallucinations in LLM’s outputs result in insufficient accuracy. To address these limitations, we propose a zero-shot, self-improving NER method for CTI via knowledge injection. The framework consists of four modules: a domain knowledge extractor, a reliable data annotator, a high-consistency annotation filter, and a self-retrieval reasoner. The domain knowledge extractor enhances LLM comprehension of specialized threat intelligence, while the others work in a multi-stage reasoning process to mitigate hallucinations by generating, filtering, and reasoning upon high-consistency data. These modules collaborate to improve the model’s entity recognition ability through continuous in-context learning. Experimental results show that under strict zero-shot conditions, the proposed method achieves F1 scores of 67.7%, 61.41%, 74.56%, and 65.83% on the LLM-TIKG, APT-NER, LADDER, and CDTier datasets, respectively. This represents an improvement of 7.66% over the average F1 score of other baseline methods, demonstrating superior adaptability in low-resource security scenarios.