标题： 用于锐度感知最小化的数据 Attribution 显示英文标题

标题： Attributing Data for Sharpness-Aware Minimization

摘要： 锐度感知最小化（SAM）通过将损失景观几何与泛化联系起来，提高了大规模模型训练的泛化能力。 然而，诸如标签错误的噪声数据和隐私问题等挑战已成为重大问题。 数据归因，即识别特定训练样本的贡献，提供了一个有前景的解决方案。 然而，直接将现有的数据影响评估工具如影响函数（IF）应用于SAM将是不可行或不准确的，因为SAM使用一个内部循环来寻找使损失最大化的模型扰动，然后外部循环对其进行最小化，导致计算结构加倍。 此外，这种双层结构使得对数据影响建模变得复杂。 在本文中，基于影响函数（IF），我们为SAM开发了两种创新的数据估值方法，在不同场景中各具独特优势：基于Hessian的影响函数和基于梯度轨迹的影响函数。 第一个方法使用仅依赖于训练后模型权重的闭式度量来全面估计数据影响。 相比之下，SAM的另一种影响函数利用训练期间的梯度轨迹信息进行更准确和高效的数据评估。 大量实验证明了它们在数据评估和参数调整中的有效性，应用包括识别标签错误的数据、模型编辑和增强可解释性。 显示英文摘要

摘要： Sharpness-aware Minimization (SAM) improves generalization in large-scale model training by linking loss landscape geometry to generalization. However, challenges such as mislabeled noisy data and privacy concerns have emerged as significant issues. Data attribution, which identifies the contributions of specific training samples, offers a promising solution. However, directly rendering existing data influence evaluation tools such as influence functions (IF) to SAM will be inapplicable or inaccurate as SAM utilizes an inner loop to find model perturbations that maximize loss, which the outer loop then minimizes, resulting in a doubled computational structure. Additionally, this bilevel structure complicates the modeling of data influence on the parameters. In this paper, based on the IF, we develop two innovative data valuation methods for SAM, each offering unique benefits in different scenarios: the Hessian-based IF and the Gradient Trajectory-based IF. The first one provides a comprehensive estimation of data influence using a closed-form measure that relies only on the trained model weights. In contrast, the other IF for SAM utilizes gradient trajectory information during training for more accurate and efficient data assessment. Extensive experiments demonstrate their effectiveness in data evaluation and parameter tuning, with applications in identifying mislabeled data, model editing, and enhancing interpretability.