标题： 标记增强生成：帮助语言模型在长上下文中找到复杂知识 显示英文标题

标题： Tagging-Augmented Generation: Assisting Language Models in Finding Intricate Knowledge In Long Contexts

摘要： 最近对现代旗舰大型语言模型（LLMs）的有效上下文长度的研究揭示了即使是最强大和令人印象深刻的模型在处理长而复杂的上下文时，在有效的问题回答（QA）和推理方面存在重大限制。 虽然像检索增强生成（RAG）和基于块的重新排序等方法试图缓解这个问题，但它们对分块、嵌入和检索策略及模型敏感，并且进一步依赖于大量的预处理、知识获取和索引步骤。 在本文中，我们提出 标签增强生成（TAG），一种轻量级的数据增强策略，在不破坏和改变检索文档的完整性和组成的情况下，提升了LLM在长上下文场景中的性能。 我们通过增强两个具有挑战性和直接相关的问答基准——NoLima和NovelQA——来验证我们的假设，并表明对上下文进行标记或甚至仅在QA提示中添加标签定义都会带来相对于基线的一致性能提升——对于32K标记的上下文最高可达17%，对于需要跨广泛文本知识的多跳查询的复杂推理问答任务则提高2.9%。 更多细节请访问 https://sites.google.com/view/tag-emnlp. 显示英文摘要

摘要： Recent investigations into effective context lengths of modern flagship large language models (LLMs) have revealed major limitations in effective question answering (QA) and reasoning over long and complex contexts for even the largest and most impressive cadre of models. While approaches like retrieval-augmented generation (RAG) and chunk-based re-ranking attempt to mitigate this issue, they are sensitive to chunking, embedding and retrieval strategies and models, and furthermore, rely on extensive pre-processing, knowledge acquisition and indexing steps. In this paper, we propose Tagging-Augmented Generation (TAG), a lightweight data augmentation strategy that boosts LLM performance in long-context scenarios, without degrading and altering the integrity and composition of retrieved documents. We validate our hypothesis by augmenting two challenging and directly relevant question-answering benchmarks -- NoLima and NovelQA -- and show that tagging the context or even just adding tag definitions into QA prompts leads to consistent performance gains over the baseline -- up to 17% for 32K token contexts, and 2.9% in complex reasoning question-answering for multi-hop queries requiring knowledge across a wide span of text. Additional details are available at https://sites.google.com/view/tag-emnlp.