标题： 基于思维链的合成器：通过答案合成提升LLM性能 显示英文标题

标题： CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis

摘要： 当前的推理缩放方法，如自一致性方法和最佳N个方法，在提高大型语言模型在复杂推理任务上的准确性方面已被证明是有效的。 然而，这些方法高度依赖于候选响应的质量，并且当所有候选响应都不正确时，无法生成正确的答案。 在本文中，我们提出了一种新的推理缩放策略，基于思维链的合成器，该策略利用思维链推理通过分析多个候选响应中的互补信息来合成更优的答案，即使所有候选响应都有缺陷。 为了实现轻量级且成本效益高的实现，我们引入了一个自动化的数据生成管道，以创建多样化的训练数据。 这使得在这些数据上训练的小型语言模型可以提升大型模型（包括基于API的语言模型）的推理准确性。 在四个基准数据集上使用七种策略模型的实验结果表明，我们的方法显著提升了性能，在MATH数据集上，Llama3-8B的提升达到11.8%，GPT-4o的提升达到10.3%。 相应的训练数据和代码已在https://github.com/RUCKBReasoning/CoT-based-Synthesizer公开。 显示英文摘要

摘要： Current inference scaling methods, such as Self-consistency and Best-of-N, have proven effective in improving the accuracy of LLMs on complex reasoning tasks. However, these methods rely heavily on the quality of candidate responses and are unable to produce correct answers when all candidates are incorrect. In this paper, we propose a novel inference scaling strategy, CoT-based Synthesizer, which leverages CoT reasoning to synthesize superior answers by analyzing complementary information from multiple candidate responses, even when all candidate responses are flawed. To enable a lightweight and cost-effective implementation, we introduce an automated data generation pipeline that creates diverse training data. This allows smaller LLMs trained on this data to improve the inference accuracy of larger models, including API-based LLMs. Experimental results across four benchmark datasets with seven policy models demonstrate that our method significantly enhances performance, with gains of 11.8% for Llama3-8B and 10.3% for GPT-4o on the MATH dataset. The corresponding training data and code are publicly available on https://github.com/RUCKBReasoning/CoT-based-Synthesizer.