标题： 大型语言模型评估中的置信度：一种针对小样本挑战的贝叶斯方法 显示英文标题

标题： Confidence in Large Language Model Evaluation: A Bayesian Approach to Limited-Sample Challenges

摘要： 大型语言模型（LLMs）表现出概率输出特性，但传统的评估框架依赖于确定性标量指标。 本研究引入了一种用于LLM能力评估的贝叶斯方法，通过概率推断整合先验知识，在有限样本情况下解决了局限性。 通过将模型能力视为潜在变量，并利用精心挑选的查询集来引发区分性响应，我们将模型排序形式化为在互斥能力区间上的贝叶斯假设检验问题。 与GPT系列模型的实验评估表明，所提出的方法相比传统评估方法具有更好的区分能力。 结果表明，即使样本数量减少，该方法仍保持统计稳健性，同时提供可操作的见解，例如关于模型超越特定基线的可能性的概率陈述。 本研究通过将贝叶斯推断与实际部署场景中的约束相结合，推进了LLM评估方法学的发展。 显示英文摘要

摘要： Large language models (LLMs) exhibit probabilistic output characteristics, yet conventional evaluation frameworks rely on deterministic scalar metrics. This study introduces a Bayesian approach for LLM capability assessment that integrates prior knowledge through probabilistic inference, addressing limitations under limited-sample regimes. By treating model capabilities as latent variables and leveraging a curated query set to induce discriminative responses, we formalize model ranking as a Bayesian hypothesis testing problem over mutually exclusive capability intervals. Experimental evaluations with GPT-series models demonstrate that the proposed method achieves superior discrimination compared to conventional evaluation methods. Results indicate that even with reduced sample sizes, the approach maintains statistical robustness while providing actionable insights, such as probabilistic statements about a model's likelihood of surpassing specific baselines. This work advances LLM evaluation methodologies by bridging Bayesian inference with practical constraints in real-world deployment scenarios.