标题： 基于CoT的综合器：通过答案合成提升LLM性能 显示英文标题

标题： CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis

摘要： 当前的推理扩展方法，例如自一致性（Self-consistency）和最佳候选（Best-of-N），已被证明在提高大型语言模型（LLMs）在复杂推理任务上的准确性方面非常有效。然而，这些方法严重依赖于候选响应的质量，并且当所有候选响应都不正确时，无法生成正确的答案。 在本文中，我们提出了一种新的推理扩展策略——基于CoT的综合器（CoT-based Synthesizer），它利用CoT推理通过分析多个候选响应中的互补信息来合成更优的答案，即使所有候选响应都存在缺陷。为了实现轻量级和成本效益的实施，我们引入了一个自动数据生成管道，以创建多样化的训练数据。这使得在该数据上训练的小型LLMs能够提升较大模型（包括基于API的LLMs）的推理准确性。 在四个基准数据集和七种策略模型的实验结果显示，我们的方法显著提高了性能，在MATH数据集上Llama3-8B的提升幅度为11.8%，GPT-4o的提升幅度为10.3%。相应的训练数据和代码已公开发布在https://github.com/RUCKBReasoning/CoT-based-Synthesizer上。 显示英文摘要

摘要： Current inference scaling methods, such as Self-consistency and Best-of-N, have proven effective in improving the accuracy of LLMs on complex reasoning tasks. However, these methods rely heavily on the quality of candidate responses and are unable to produce correct answers when all candidates are incorrect. In this paper, we propose a novel inference scaling strategy, CoT-based Synthesizer, which leverages CoT reasoning to synthesize superior answers by analyzing complementary information from multiple candidate responses, even when all candidate responses are flawed. To enable a lightweight and cost-effective implementation, we introduce an automated data generation pipeline that creates diverse training data. This allows smaller LLMs trained on this data to improve the inference accuracy of larger models, including API-based LLMs. Experimental results across four benchmark datasets with seven policy models demonstrate that our method significantly enhances performance, with gains of 11.8% for Llama3-8B and 10.3% for GPT-4o on the MATH dataset. The corresponding training data and code are publicly available on https://github.com/RUCKBReasoning/CoT-based-Synthesizer.