标题： 带有隐藏状态的差分隐私零阶优化中的隐私放大 显示英文标题

标题： Privacy Amplification in Differentially Private Zeroth-Order Optimization with Hidden States

摘要： 零阶优化作为一种在领域特定数据上微调大型语言模型的有前景的方法，在差分隐私（DP）和内存限制下尤其如此。 尽管一阶方法从隐私的角度已经得到了广泛的研究，但零阶方法的隐私分析和算法设计仍然显著缺乏探索。 一个关键的开放性问题是隐藏状态的DP分析：虽然一阶方法已知具有收敛的隐私界值，但对于零阶方法是否可以建立类似的保证仍不清楚。 在这项工作中，我们通过证明零阶优化的收敛DP界给出了肯定的答案。 我们的分析将著名的迭代隐私放大框架推广到了零阶优化中平滑损失函数的情况下。 此外，它还产生了更好的DP零阶算法设计，这些设计之前在文献中是未知的。 显示英文摘要

摘要： Zeroth-order optimization has emerged as a promising approach for fine-tuning large language models on domain-specific data, particularly under differential privacy (DP) and memory constraints. While first-order methods have been extensively studied from a privacy perspective, the privacy analysis and algorithmic design for zeroth-order methods remain significantly underexplored. A critical open question concerns hidden-state DP analysis: although convergent privacy bounds are known for first-order methods, it has remained unclear whether similar guarantees can be established for zeroth-order methods. In this work, we provide an affirmative answer by proving a convergent DP bound for zeroth-order optimization. Our analysis generalizes the celebrated privacy amplification-by-iteration framework to the setting of smooth loss functions in zeroth-order optimization. Furthermore, it induces better DP zeroth-order algorithmic designs that are previously unknown to the literature.