Title: kTULA: A Langevin sampling algorithm with improved KL bounds under super-linear log-gradients Show Chinese title

Title: kTULA：一种具有改进的超线性对数梯度KL界的Langevin抽样算法

Abstract: Motivated by applications in deep learning, where the global Lipschitz continuity condition is often not satisfied, we examine the problem of sampling from distributions with super-linearly growing log-gradients. We propose a novel tamed Langevin dynamics-based algorithm, called kTULA, to solve the aforementioned sampling problem, and provide a theoretical guarantee for its performance. More precisely, we establish a non-asymptotic convergence bound in Kullback-Leibler (KL) divergence with the best-known rate of convergence equal to $2-\overline{\epsilon}$, $\overline{\epsilon}>0$, which significantly improves relevant results in existing literature. This enables us to obtain an improved non-asymptotic error bound in Wasserstein-2 distance, which can be used to further derive a non-asymptotic guarantee for kTULA to solve the associated optimization problems. To illustrate the applicability of kTULA, we apply the proposed algorithm to the problem of sampling from a high-dimensional double-well potential distribution and to an optimization problem involving a neural network. We show that our main results can be used to provide theoretical guarantees for the performance of kTULA. Show Chinese abstract

Abstract: 受深度学习应用的启发，其中全局Lipschitz连续性条件通常不满足，我们研究了从具有超线性增长的对数梯度的分布中采样的问题。我们提出了一种基于改进型 Langevin 动力学的新算法，称为kTULA，以解决上述采样问题，并为其性能提供了理论保证。更具体地说，我们在 Kullback-Leibler (KL) 散度中建立了非渐近收敛界，其最佳收敛率等于 $2-\overline{\epsilon}$, $\overline{\epsilon}>0$，这显著改进了现有文献中的相关结果。这使我们能够在 Wasserstein-2 距离中获得改进的非渐近误差界，该界可以进一步用于推导出kTULA解决相关优化问题的非渐近保证。为了展示kTULA的适用性，我们将所提出的算法应用于从高维双井势分布中采样的问题以及涉及神经网络的优化问题。我们表明，我们的主要结果可以用于提供kTULA性能的理论保证。