标题： 通过可能性的视角重新定义大型语言模型的数据价值 显示英文标题

标题： Reframing Data Value for Large Language Models Through the Lens of Plausibility

摘要： 数据估值旨在回答一个重要问题：“这些数据的价值是多少？” 现有的数据估值方法主要集中在判别模型上，主要通过数据在训练中的效用来衡量其价值。然而，随着大规模语言模型的兴起，依赖需要训练的估值方法变得越来越昂贵，并且更加依赖特定技术。 我们为语言模型的数据价值问题提出了一种替代视角，围绕数据的可能性展开讨论。我们认为，如果数据可以被模型本身合理生成，那么它的价值就较低。 从一些符合我们对有价值数据直觉的标准出发，我们开发了一种计算可行且基于第一性原理的新颖价值函数，该函数具有可证明的特性。我们对该价值函数进行了理论分析，并在多个场景和数据集上对其进行了评估。 显示英文摘要

摘要： Data valuation seeks to answer the important question, "How much is this data worth?" Existing data valuation methods have largely focused on discriminative models, primarily examining data value through the lens of its utility in training. However, with the push for ever-larger language models, relying on valuation methods that require training becomes increasingly expensive and dependent on specific techniques. We propose an alternative perspective on the data value problem for language models, centering around the plausibility of the data. We posit that data holds lesser value if it can be plausibly generated by the model itself. Starting from some intuitive criteria that align with our notions of valuable data, we develop a novel value function that is computationally tractable and derived from first principles with provable properties. We conduct a theoretical analysis of our value function and evaluate it across multiple scenarios and datasets.