标题： 检索增强生成作为噪声上下文学习：一个统一的理论和风险界 显示英文标题

标题： Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Bounds

摘要： 基于检索的生成（RAG）近年来通过辅助大型语言模型（LLM）获取外部知识取得了许多经验上的成功。然而，其理论方面仍未得到充分探索。本文首次提出了针对上下文线性回归中的RAG的有限样本广义界，并推导出精确的偏差-方差权衡。我们的框架将检索到的文本视为与查询相关的噪声上下文示例，并将经典的上下文学习（ICL）和标准RAG作为极限情况恢复。我们的分析表明，与ICL相比，RAG存在一个内在的泛化误差上限。此外，通过引入均匀和非均匀RAG噪声，我们的框架能够建模从训练数据和外部语料库中进行检索。与我们的理论一致，我们通过在常见问答基准（如Natural Questions和TriviaQA）上的实验，实证展示了ICL和RAG的样本效率。 显示英文摘要

摘要： Retrieval-augmented generation (RAG) has seen many empirical successes in recent years by aiding the LLM with external knowledge. However, its theoretical aspect has remained mostly unexplored. In this paper, we propose the first finite-sample generalization bound for RAG in in-context linear regression and derive an exact bias-variance tradeoff. Our framework views the retrieved texts as query-dependent noisy in-context examples and recovers the classical in-context learning (ICL) and standard RAG as the limit cases. Our analysis suggests that an intrinsic ceiling on generalization error exists on RAG as opposed to the ICL. Furthermore, our framework is able to model retrieval both from the training data and from external corpora by introducing uniform and non-uniform RAG noise. In line with our theory, we show the sample efficiency of ICL and RAG empirically with experiments on common QA benchmarks, such as Natural Questions and TriviaQA.