Application of retrieval-augmented generation for interactive industrial knowledge management via a large language model
Lun-Chi Chen, Mayuresh Sunil Pardeshi, Y Liao, Kai-Chih Pai
Abstract
• To design a document knowledge retrieval system for the domain specific data using RAG and LLM. • To model an effective approach for selecting high score chunks. • Rank and respond to the complex queries for the product management. • To evaluate the design system model in detail with the experience. Industrial data processing and retrieval are necessary for adoption in Industry 5.0. Large Language Model (LLMs) revolutionize natural language process (NLP) but face challenges in domain-specific applications due to specialized terminology and context. Artificial Intelligence (AI) assistants for industrial-related work enquiry and customer support services are necessary for increasing demand and quality of service (QoS). Our research aims to design a novel customized model with a retrieval-augmented generation (RAG)-based LLM as a sustainable solution for industrial integration with AI. The goal is to provide an interactive industrial knowledge management (IIKM) system that can be applied to technical services: assisting technicians in the search for precise technical repair details and company internal regulation searches: personnel can easily inquire about regulations, such as business trips and leave requirements. The IIKM model architecture consists of BM25 and embedding sequence processing in the chroma database, where the top k-chunks are selected by the BAAI ranker to respond effectively to the queries. A group of documents of 234 MB size and pdf, pptx, docx, csv and txt formats are used for the experimental analysis. The designed interactive knowledge management system has a mean reciprocal rank (MRR) of 88 %, a recall of 85 % and a mean average precision (mAP) of 75 % in technical service. The internal regulatory documents have a generation-based retrieval evaluation prediction of recall of 91.62 %, MRR of 97.97 % and mAP of 91.12 %. We conclude with insights gained and experiences shared from IIKM deployment with Sakura incorporation, highlighting the importance of the hybrid approach integrating RAG-based generative pretrained transformer (GPT) models for customized solutions.