标题： ToolComp：多工具推理与过程监督基准 显示英文标题

标题： ToolComp: A Multi-Tool Reasoning & Process Supervision Benchmark

摘要： 尽管人工智能取得了最近的进展，但开发能够执行涉及多个工具的复杂多步骤推理任务的系统仍然是一个重大挑战。 当前的基准测试在捕捉工具使用推理的真实世界复杂性方面存在不足，其中验证最终答案以及中间步骤的正确性对于评估、开发和识别推理过程中的失败至关重要。 为了弥补这一差距，我们引入了ToolComp，这是一个旨在评估多步骤工具使用推理的全面基准。 ToolComp是通过模型和人类标注者的合作开发的，包括经过人工编辑/验证的提示、最终答案和过程监督标签，从而可以对最终结果和中间推理进行评估。 对六个不同模型系列的评估表明了我们数据集的挑战性，大多数模型的准确率不到50%。 此外，我们生成了合成训练数据，以比较结果监督奖励模型（ORMs）与过程监督奖励模型（PRMs）的性能，以评估它们在ToolComp评估下提高复杂工具使用推理能力的能力。 我们的结果表明，PRMs在泛化能力上明显优于ORMs，分别在排名基线模型和微调模型轨迹的rank@1准确率上提高了19%和11%。 这些发现突显了过程监督在人工智能模型的评估和训练中的关键作用，为在复杂多步骤工具使用任务中构建更强大和可靠系统铺平了道路。 显示英文摘要

摘要： Despite recent advances in AI, the development of systems capable of executing complex, multi-step reasoning tasks involving multiple tools remains a significant challenge. Current benchmarks fall short in capturing the real-world complexity of tool-use reasoning, where verifying the correctness of not only the final answer but also the intermediate steps is important for evaluation, development, and identifying failures during inference time. To bridge this gap, we introduce ToolComp, a comprehensive benchmark designed to evaluate multi-step tool-use reasoning. ToolComp is developed through a collaboration between models and human annotators, featuring human-edited/verified prompts, final answers, and process supervision labels, allowing for the evaluation of both final outcomes and intermediate reasoning. Evaluation across six different model families demonstrates the challenging nature of our dataset, with the majority of models achieving less than 50% accuracy. Additionally, we generate synthetic training data to compare the performance of outcome-supervised reward models (ORMs) with process-supervised reward models (PRMs) to assess their ability to improve complex tool-use reasoning as evaluated by ToolComp. Our results show that PRMs generalize significantly better than ORMs, achieving a 19% and 11% improvement in rank@1 accuracy for ranking base and fine-tuned model trajectories, respectively. These findings highlight the critical role of process supervision in both the evaluation and training of AI models, paving the way for more robust and capable systems in complex, multi-step tool-use tasks.