标题： 在基于比较的偏好学习模型中推广并保持单调性 显示英文标题

标题： Generalizing while preserving monotonicity in comparison-based preference learning models

摘要： 如果你告诉一个学习模型你更喜欢备选项$a$而不是另一个备选项$b$，那么你可能期望该模型是单调的，也就是说，$a$的估值增加，而$b$的估值减少。 然而，也许令人惊讶的是，许多广泛部署的基于比较的偏好学习模型，包括大型语言模型，都无法保证这一点。 直到现在，唯一被证明具有单调性的基于比较的偏好学习算法是广义布拉德利-特雷模型（Generalized Bradley-Terry 模型）。 然而，这些模型无法推广到未比较的数据。 在本文中，我们深入研究了具备泛化能力且单调的模型集合。 具体来说，我们提出了一类新的带有扩散先验的线性广义布拉德利-特雷模型，并确定了关于备选项嵌入的充分条件，以确保单调性。 我们的实验表明，这种单调性远非普遍保证，并且我们提出的这一类能够推广的新模型提高了准确性，尤其是在数据集有限的情况下。 显示英文摘要

摘要： If you tell a learning model that you prefer an alternative $a$ over another alternative $b$, then you probably expect the model to be monotone, that is, the valuation of $a$ increases, and that of $b$ decreases. Yet, perhaps surprisingly, many widely deployed comparison-based preference learning models, including large language models, fail to have this guarantee. Until now, the only comparison-based preference learning algorithms that were proved to be monotone are the Generalized Bradley-Terry models. Yet, these models are unable to generalize to uncompared data. In this paper, we advance the understanding of the set of models with generalization ability that are monotone. Namely, we propose a new class of Linear Generalized Bradley-Terry models with Diffusion Priors, and identify sufficient conditions on alternatives' embeddings that guarantee monotonicity. Our experiments show that this monotonicity is far from being a general guarantee, and that our new class of generalizing models improves accuracy, especially when the dataset is limited.