Pencils Down! Automatic Rubric-based Evaluation of Retrieve/Generate Systems
Naghmeh Farzi, Laura Dietz
Abstract
Current IR evaluation paradigms are challenged by large language models (LLMs) and retrieval-augmented generation (RAG) methods. Furthermore, evaluation either resorts to expensive human judgments or lead to an over-reliance on LLMs.
Topics & Concepts
RubricComputer scienceNatural language processingInformation retrievalArtificial intelligencePsychologyMathematics educationTopic ModelingMachine Learning in Materials ScienceAdvanced Graph Neural Networks