Robust Multi Model RAG Pipeline For Documents Containing Text, Table & Images
Pankaj Joshi, Aditya Gupta, Pankaj Kumar, M. S. Sisodia
Abstract
RAG (Retrieval Augmented Generation) is generally used for generating results from the existing knowledge-base. RAG refers to finding references (R), Adding references (A) and improving generation(i.e, answers to the question) (G). MultiModel-RAGs are used for generation of results over the documents which contain images and texts. There exists multiple different Multimodel-RAGs but these are not still efficient in generation of the results from the documents which contain relationships between images and texts. This study has proposed the solution to enable effective retrieval and generation of results, which includes the relationship between images and texts. The comparison of proposed Multimodal RAG with four different datasets (i.e., Short-form-type-QA, Long-form-type-QA, MCQ-type-QA, True-False-type-QA) shows the proposed solution improves the effectiveness of the existing Multimodal RAGs. Testing of proposed Multimodal RAG over two different other multimodal LLM i.e, Open-AI & Gemini helps in deciding whether the proposed solution fits best with LLM in different cases.