标题： DynamixSFT：指令微调数据集的动态混合优化 显示英文标题

标题： DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections

摘要： 随着越来越多的指令微调数据集在训练后阶段出现，动态平衡和优化它们的混合变得是一个关键挑战。 为了解决这个问题，我们提出了DynamixSFT，一种用于指令微调数据集混合优化的动态和自动化方法。 我们将问题形式化为一个多臂老虎机设置，并引入了一种先验缩放的玻尔兹曼探索，该方法将更新后的采样分布软性地锚定到原始数据集比例，从而保留了数据集集合的固有多样性和覆盖范围。 使用轻量级的1步前瞻奖励来更新采样概率，反映数据集对当前状态下模型性能提升的贡献程度。 当应用于包含16个指令微调数据集的Tulu-v2-mixture集合时，DynamixSFT在10个基准测试中实现了高达2.2%的性能提升。 此外，我们提供了全面的分析和可视化，以更深入地了解我们方法的自适应动态。 显示英文摘要

摘要： As numerous instruction-tuning datasets continue to emerge during the post-training stage, dynamically balancing and optimizing their mixtures has become a critical challenge. To address this, we propose DynamixSFT, a dynamic and automated method for instruction-tuning dataset mixture optimization. We formulate the problem as a multi-armed bandit setup and introduce a Prior-scaled Boltzmann Exploration that softly anchors the updated sampling distribution to the original dataset proportions, thereby preserving the inherent diversity and coverage of the collection. Sampling probabilities are updated using a lightweight 1-Step Look-ahead Reward, reflecting how much the dataset contributes to improving the model's performance at its current state. When applied to the Tulu-v2-mixture collection comprising 16 instruction-tuning datasets, DynamixSFT achieves up to a 2.2% performance improvement across 10 benchmarks. Furthermore, we provide a comprehensive analysis and visualizations to offer deeper insights into the adaptive dynamics of our method.