标题： 强化学习辅助的大模型集成 (RLAE) 显示英文标题

标题： RLAE: Reinforcement Learning-Assisted Ensemble for LLMs

摘要： 集成大型语言模型（LLMs）能够有效结合不同模型的多样性优势，为提升各种任务性能提供了一种有前景的方法。然而，现有的方法通常依赖固定的加权策略，这些策略无法适应LLMs能力的动态、上下文相关的特性。 在这项工作中，我们提出了基于强化学习辅助的大语言模型集成框架（RLAE），这是一个通过马尔可夫决策过程（MDP）视角重新定义LLM集成的新框架。我们的方法引入了一个强化学习代理，该代理通过考虑输入上下文和中间生成状态来动态调整集成权重，并使用直接对应于最终输出质量的奖励进行训练。 我们使用单代理和多代理强化学习算法（$\text{RLAE}_\text{PPO}$和$\text{RLAE}_\text{MAPPO}$）实现了RLAE，在多种任务上显著优于传统的集成方法。广泛的评估显示，RLAE在一组多样化任务中的表现比现有方法高出$3.3\%$准确率点，提供了一个更有效的LLM集成框架。此外，我们的方法在不同任务中表现出卓越的泛化能力，无需重新训练即可实现更低的时间延迟。 显示英文摘要

摘要： Ensembling large language models (LLMs) can effectively combine diverse strengths of different models, offering a promising approach to enhance performance across various tasks. However, existing methods typically rely on fixed weighting strategies that fail to adapt to the dynamic, context-dependent characteristics of LLM capabilities. In this work, we propose Reinforcement Learning-Assisted Ensemble for LLMs (RLAE), a novel framework that reformulates LLM ensemble through the lens of a Markov Decision Process (MDP). Our approach introduces a RL agent that dynamically adjusts ensemble weights by considering both input context and intermediate generation states, with the agent being trained using rewards that directly correspond to the quality of final outputs. We implement RLAE using both single-agent and multi-agent reinforcement learning algorithms ($\text{RLAE}_\text{PPO}$ and $\text{RLAE}_\text{MAPPO}$ ), demonstrating substantial improvements over conventional ensemble methods. Extensive evaluations on a diverse set of tasks show that RLAE outperforms existing approaches by up to $3.3\%$ accuracy points, offering a more effective framework for LLM ensembling. Furthermore, our method exhibits superior generalization capabilities across different tasks without the need for retraining, while simultaneously achieving lower time latency.