标题： 梯度算法在非凸C^{1+alpha}代价函数中的收敛性 显示英文标题

标题： Convergence of Gradient Algorithms for Nonconvex C^{1+alpha} Cost Functions

摘要： 本文研究了在非凸设置下带有动量项的随机梯度算法的收敛性。 在较弱的假设下，分析了一类随机动量方法，包括随机梯度下降、heavy ball方法和Nesterov加速梯度方法。 基于期望梯度的收敛结果，通过详细讨论动量和穿越次数的影响，证明了几乎必然收敛。 值得注意的是，对目标函数和步长没有额外的限制。 另一个改进是，现有的梯度Lipschitz条件被放松为Hölder连续性条件。 作为副产品，我们应用了一个局部化过程，将我们的结果扩展到随机步长。 显示英文摘要

摘要： This paper is concerned with convergence of stochastic gradient algorithms with momentum terms in the nonconvex setting. A class of stochastic momentum methods, including stochastic gradient descent, heavy ball, and Nesterov's accelerated gradient, is analyzed in a general framework under mild assumptions. Based on the convergence result of expected gradients, we prove the almost sure convergence by a detailed discussion of the effects of momentum and the number of upcrossings. It is worth noting that there are not additional restrictions imposed on the objective function and stepsize. Another improvement over previous results is that the existing Lipschitz condition of the gradient is relaxed into the condition of Holder continuity. As a byproduct, we apply a localization procedure to extend our results to stochastic stepsizes.