标题： 面向领域偏移的大型语言模型共形预测 显示英文标题

标题： Domain-Shift-Aware Conformal Prediction for Large Language Models

摘要： 大型语言模型在各种任务中取得了令人印象深刻的性能。 然而，它们产生过于自信且事实错误的输出的倾向，即所谓的幻觉，在现实世界的应用中带来了风险。 置信预测提供了有限样本、分布无关的覆盖保证，但在领域转移下标准的置信预测会失效，通常导致覆盖不足和不可靠的预测集。 我们提出了一种新的框架，称为领域转移感知置信预测（DS-CP）。 我们的框架通过系统地重新加权校准样本，根据它们与测试提示的接近程度，使置信预测适应于领域转移下的大型语言模型，从而在保持有效性的同时增强适应性。 我们的理论分析和在MMLU基准上的实验表明，所提出的方法在大量分布转移下比标准置信预测提供了更可靠的覆盖，同时保持了效率。 这为在实际部署中实现大型语言模型的可信不确定性量化提供了一个实用的步骤。 显示英文摘要

摘要： Large language models have achieved impressive performance across diverse tasks. However, their tendency to produce overconfident and factually incorrect outputs, known as hallucinations, poses risks in real world applications. Conformal prediction provides finite-sample, distribution-free coverage guarantees, but standard conformal prediction breaks down under domain shift, often leading to under-coverage and unreliable prediction sets. We propose a new framework called Domain-Shift-Aware Conformal Prediction (DS-CP). Our framework adapts conformal prediction to large language models under domain shift, by systematically reweighting calibration samples based on their proximity to the test prompt, thereby preserving validity while enhancing adaptivity. Our theoretical analysis and experiments on the MMLU benchmark demonstrate that the proposed method delivers more reliable coverage than standard conformal prediction, especially under substantial distribution shifts, while maintaining efficiency. This provides a practical step toward trustworthy uncertainty quantification for large language models in real-world deployment.