标题： MuRating：一种多语言大型语言模型预训练的高质量数据选择方法 显示英文标题

标题： MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining

摘要： 数据质量是大型语言模型性能的关键驱动因素，但现有的基于模型的选择方法几乎只关注英语。 我们引入了MuRating，一个可扩展的框架，将高质量的英语数据质量信号转移到17种目标语言的单一评分者中。 MuRating通过成对比较聚合多个英语“评分者”以学习统一的文档质量分数，然后通过翻译将这些判断投影出来，以在单语、跨语言和并行文本对上训练多语言评估者。 应用于网络数据时，MuRating选择了平衡的英语和多语言内容子集来预训练一个12亿参数的LLaMA模型。 与包括QuRater、AskLLM、DCLM等在内的强基线相比，我们的方法在英语基准和多语言评估中的平均准确率都有所提高，特别是在知识密集型任务中取得了更大的进展。 我们进一步分析了翻译保真度、选择偏差和叙述性材料的不足代表，指出了未来工作的方向。 显示英文摘要

摘要： Data quality is a critical driver of large language model performance, yet existing model-based selection methods focus almost exclusively on English. We introduce MuRating, a scalable framework that transfers high-quality English data-quality signals into a single rater for 17 target languages. MuRating aggregates multiple English "raters" via pairwise comparisons to learn unified document-quality scores,then projects these judgments through translation to train a multilingual evaluator on monolingual, cross-lingual, and parallel text pairs. Applied to web data, MuRating selects balanced subsets of English and multilingual content to pretrain a 1.2 B-parameter LLaMA model. Compared to strong baselines, including QuRater, AskLLM, DCLM and so on, our approach boosts average accuracy on both English benchmarks and multilingual evaluations, with especially large gains on knowledge-intensive tasks. We further analyze translation fidelity, selection biases, and underrepresentation of narrative material, outlining directions for future work.