标题： 评估大型语言模型对先验上下文的敏感性 显示英文标题

标题： Evaluating the Sensitivity of LLMs to Prior Context

摘要： 随着大型语言模型（LLMs）越来越多地部署在多轮对话和其他持续交互场景中，理解扩展上下文对其性能的影响至关重要。专注于单轮问答（QA）任务的流行基准无法捕捉多轮交互的影响。为了解决这一差距，我们引入了一组新的基准，系统地改变先前上下文的数量和性质。我们在这些基准上评估了多个传统的LLMs，包括GPT、Claude和Gemini，以测量它们对上下文变化的敏感性。我们的研究结果显示，在多轮交互中，LLM在多项选择题上的表现可能会大幅下降，某些模型的表现降幅高达73%。即使是能力极强的模型，如GPT-4o，其准确率也会下降多达32%。值得注意的是，较大模型与较小模型之间的相对性能并不总是可预测的。此外，合理安排任务描述在上下文中的位置可以显著减轻性能下降，提高准确性最多达3.5倍。这些发现强调了需要稳健的策略来设计、评估和缓解LLMs的上下文相关敏感性。 显示英文摘要

摘要： As large language models (LLMs) are increasingly deployed in multi-turn dialogue and other sustained interactive scenarios, it is essential to understand how extended context affects their performance. Popular benchmarks, focusing primarily on single-turn question answering (QA) tasks, fail to capture the effects of multi-turn exchanges. To address this gap, we introduce a novel set of benchmarks that systematically vary the volume and nature of prior context. We evaluate multiple conventional LLMs, including GPT, Claude, and Gemini, across these benchmarks to measure their sensitivity to contextual variations. Our findings reveal that LLM performance on multiple-choice questions can degrade dramatically in multi-turn interactions, with performance drops as large as 73% for certain models. Even highly capable models such as GPT-4o exhibit up to a 32% decrease in accuracy. Notably, the relative performance of larger versus smaller models is not always predictable. Moreover, the strategic placement of the task description within the context can substantially mitigate performance drops, improving the accuracy by as much as a factor of 3.5. These findings underscore the need for robust strategies to design, evaluate, and mitigate context-related sensitivity in LLMs.