计算机科学
一套
背景(考古学)
利用
情报检索
基本事实
人工智能
计算机安全
历史
考古
作者
S Davies-van Es,Jithin James,Luis Espinosa-Anke,Steven Schockaert
出处
期刊:Cornell University - arXiv
日期:2023-01-01
被引量:32
标识
DOI:10.48550/arxiv.2309.15217
摘要
We introduce RAGAs (Retrieval Augmented Generation Assessment), a framework for reference-free evaluation of Retrieval Augmented Generation (RAG) pipelines. RAG systems are composed of a retrieval and an LLM based generation module, and provide LLMs with knowledge from a reference textual database, which enables them to act as a natural language layer between a user and textual databases, reducing the risk of hallucinations. Evaluating RAG architectures is, however, challenging because there are several dimensions to consider: the ability of the retrieval system to identify relevant and focused context passages, the ability of the LLM to exploit such passages in a faithful way, or the quality of the generation itself. With RAGAs, we put forward a suite of metrics which can be used to evaluate these different dimensions \textit{without having to rely on ground truth human annotations}. We posit that such a framework can crucially contribute to faster evaluation cycles of RAG architectures, which is especially important given the fast adoption of LLMs.
科研通智能强力驱动
Strongly Powered by AbleSci AI