标题： 覆盖原则：预训练如何促进微调 显示英文标题

标题： The Coverage Principle: How Pre-training Enables Post-Training

摘要： 语言模型在大规模文本语料库上进行预训练并在特定任务上进行微调时表现出惊人的能力，但预训练如何以及为何影响最终模型的成功仍知之甚少。值得注意的是，尽管预训练的成功通常通过交叉熵损失来量化，但交叉熵可能是下游性能的不良预测指标。相反，我们通过\emph{覆盖范围}的视角提供了一个理论观点，该指标衡量预训练模型对高质量响应的概率质量，并且对于后续训练和测试时缩放方法（如 Best-of-N）的成功是必要且充分的。我们的主要结果发展了对\emph{覆盖原则}的理解，这是一种现象，即下一个标记预测隐式地优化到具有良好覆盖性的模型。特别是，我们揭示了一种机制，解释了覆盖性在预测下游性能中的作用：\emph{覆盖率比交叉熵泛化得更快}，避免对问题相关参数（如序列长度）的虚假依赖。我们还研究了具有可证明优势的实用算法干预措施，以提高覆盖性，包括（i）模型/检查点选择程序，（ii）梯度归一化方案，以及（iii）测试时的解码策略。 显示英文摘要

摘要： Language models demonstrate remarkable abilities when pre-trained on large text corpora and fine-tuned for specific tasks, but how and why pre-training shapes the success of the final model remains poorly understood. Notably, although pre-training success is often quantified by cross entropy loss, cross-entropy can be a poor predictor of downstream performance. Instead, we provide a theoretical perspective on this relationship through the lens of \emph{coverage}, which quantifies the probability mass the pre-trained model places on high-quality responses and which is necessary and sufficient for post-training and test-time scaling methods such as Best-of-N to succeed. Our main results develop an understanding of \emph{the coverage principle}, a phenomenon whereby next-token prediction implicitly optimizes toward a model with good coverage. In particular, we uncover a mechanism that explains the power of coverage in predicting downstream performance: \emph{coverage generalizes faster than cross entropy}, avoiding spurious dependence on problem-dependent parameters such as the sequence length. We also study practical algorithmic interventions with provable benefits for improving coverage, including (i) model/checkpoint selection procedures, (ii) gradient normalization schemes, and (iii) test-time decoding strategies.