标题： 蒙蒂霍尔与优化共形预测以改进大语言模型的决策制定 显示英文标题

标题： Monty Hall and Optimized Conformal Prediction to Improve Decision-Making with LLMs

摘要： 大型语言模型（LLMs）正在增强多个应用中的决策能力，包括工具或API使用以及回答选择题（MCQs）。 然而，它们常常做出过于自信的错误预测，在医疗和金融等高风险环境中可能带来风险。 为了缓解这些风险，最近的研究已经使用了合规预测（CP），这是一种模型无关的框架，用于分布自由的不确定性量化。 CP将\emph{得分函数}转换为预测集，这些集合以高概率包含真实答案。 虽然CP对于任意得分都提供了这种覆盖保证，但得分质量显著影响预测集的大小。 先前的工作依赖于LLM的logits或其他启发式得分，缺乏质量保证。 我们通过引入CP-OPT优化框架来解决这一限制，该框架学习得分以在保持覆盖的同时最小化集合大小。 此外，受蒙提霍尔问题的启发，我们将CP的效用扩展到不确定性量化之外，以提高准确性。 我们提出了\emph{问题的共形修订}（CROQ）来通过缩小可用选项到预测集中的选项来修改问题。 CP的覆盖保证确保正确选择以高概率出现在修改后的问题提示中，而较少的选项增加了LLM正确回答的可能性。 在MMLU、ToolAlpaca和TruthfulQA数据集上，使用Gemma-2、Llama-3和Phi-3模型的实验表明，CP-OPT显著减少了集合大小，同时保持了覆盖性，并且CROQ在标准推理基础上提高了准确性，尤其是在与CP-OPT得分结合时。 总之，CP-OPT和CROQ提供了一个稳健的框架，用于提高LLM驱动决策的安全性和准确性。 显示英文摘要

摘要： Large language models (LLMs) are empowering decision-making in several applications, including tool or API usage and answering multiple-choice questions (MCQs). However, they often make overconfident, incorrect predictions, which can be risky in high-stakes settings like healthcare and finance. To mitigate these risks, recent works have used conformal prediction (CP), a model-agnostic framework for distribution-free uncertainty quantification. CP transforms a \emph{score function} into prediction sets that contain the true answer with high probability. While CP provides this coverage guarantee for arbitrary scores, the score quality significantly impacts prediction set sizes. Prior works have relied on LLM logits or other heuristic scores, lacking quality guarantees. We address this limitation by introducing CP-OPT, an optimization framework to learn scores that minimize set sizes while maintaining coverage. Furthermore, inspired by the Monty Hall problem, we extend CP's utility beyond uncertainty quantification to improve accuracy. We propose \emph{conformal revision of questions} (CROQ) to revise the problem by narrowing down the available choices to those in the prediction set. The coverage guarantee of CP ensures that the correct choice is in the revised question prompt with high probability, while the smaller number of choices increases the LLM's chances of answering it correctly. Experiments on MMLU, ToolAlpaca, and TruthfulQA datasets with Gemma-2, Llama-3 and Phi-3 models show that CP-OPT significantly reduces set sizes while maintaining coverage, and CROQ improves accuracy over the standard inference, especially when paired with CP-OPT scores. Together, CP-OPT and CROQ offer a robust framework for improving both the safety and accuracy of LLM-driven decision-making.