标题： 无需窥探的调优：LLM后训练的可证明隐私和泛化界限 显示英文标题

标题： Tuning without Peeking: Provable Privacy and Generalization Bounds for LLM Post-Training

摘要： 基于梯度的优化是深度学习的核心，通过反向传播提供高效且可扩展的训练。然而，其对大量标记数据的依赖引发了隐私和安全问题，如容易受到数据污染攻击和过拟合的风险。相比之下，黑盒优化方法将模型视为一个不透明的函数，仅依靠函数评估来指导优化，在数据访问受限、对抗风险高或过拟合是关注点的情况下，提供了有前景的替代方案。然而，黑盒方法也带来了重大挑战，包括在高维参数空间中的可扩展性差，这在大型语言模型（LLMs）中很常见，以及由于依赖大量模型评估而导致的高计算成本。本文介绍了BBoxER，这是一种用于LLM后训练的进化黑盒方法，通过隐式压缩训练数据来引入信息瓶颈。利用信息流的可处理性，我们提供了泛化、差分隐私、对数据污染攻击的易感性和对提取攻击的鲁棒性的强理论界限。BBoxER在预训练的LLMs之上运行，提供了一种轻量级且模块化的增强，适用于受限或隐私敏感环境，除了非空泛化保证外。在LLM的实验中，我们实证表明重构方法能够学习，展示了BBoxER的几次迭代如何在推理数据集基准上提高性能并具有良好的泛化能力。这使BBoxER成为基于梯度的优化的一个有吸引力的附加组件。 显示英文摘要

摘要： Gradient-based optimization is the workhorse of deep learning, offering efficient and scalable training via backpropagation. However, its reliance on large volumes of labeled data raises privacy and security concerns such as susceptibility to data poisoning attacks and the risk of overfitting. In contrast, black box optimization methods, which treat the model as an opaque function, relying solely on function evaluations to guide optimization, offer a promising alternative in scenarios where data access is restricted, adversarial risks are high, or overfitting is a concern. However, black box methods also pose significant challenges, including poor scalability to high-dimensional parameter spaces, as prevalent in large language models (LLMs), and high computational costs due to reliance on numerous model evaluations. This paper introduces BBoxER, an evolutionary black-box method for LLM post-training that induces an information bottleneck via implicit compression of the training data. Leveraging the tractability of information flow, we provide strong theoretical bounds on generalization, differential privacy, susceptibility to data poisoning attacks, and robustness to extraction attacks. BBoxER operates on top of pre-trained LLMs, offering a lightweight and modular enhancement suitable for deployment in restricted or privacy-sensitive environments, in addition to non-vacuous generalization guarantees. In experiments with LLMs, we demonstrate empirically that Retrofitting methods are able to learn, showing how a few iterations of BBoxER improve performance and generalize well on a benchmark of reasoning datasets. This positions BBoxER as an attractive add-on on top of gradient-based optimization.