标题： 基于子采样和三次正则化的更快黎曼牛顿型优化 显示英文标题

标题： Faster Riemannian Newton-type Optimization by Subsampling and Cubic Regularization

摘要： 这项工作是关于约束的大规模非凸优化，其中约束集暗示了一个流形结构。 解决这类问题在众多基础机器学习任务中非常重要。 最近在黎曼优化方面的进展使得通过在流形上适应无约束优化算法来方便地恢复解成为可能。 然而，同时提高规模并保持稳定的收敛速度以及处理鞍点仍然是一个挑战。 我们提出了一种新的二阶黎曼优化算法，旨在提高收敛速度并降低计算成本。 它增强了黎曼信任区域算法，该算法通过子采样和三次正则化技术的混合来探索曲率信息以逃出鞍点。 我们进行了严格的分析来研究所提出的算法的收敛行为。 我们还进行了广泛的实验，基于两个通用的机器学习任务使用多个数据集来评估它。 与大量最先进的黎曼优化算法相比，所提出的算法表现出更快的计算速度和更好的收敛行为。 显示英文摘要

摘要： This work is on constrained large-scale non-convex optimization where the constraint set implies a manifold structure. Solving such problems is important in a multitude of fundamental machine learning tasks. Recent advances on Riemannian optimization have enabled the convenient recovery of solutions by adapting unconstrained optimization algorithms over manifolds. However, it remains challenging to scale up and meanwhile maintain stable convergence rates and handle saddle points. We propose a new second-order Riemannian optimization algorithm, aiming at improving convergence rate and reducing computational cost. It enhances the Riemannian trust-region algorithm that explores curvature information to escape saddle points through a mixture of subsampling and cubic regularization techniques. We conduct rigorous analysis to study the convergence behavior of the proposed algorithm. We also perform extensive experiments to evaluate it based on two general machine learning tasks using multiple datasets. The proposed algorithm exhibits improved computational speed and convergence behavior compared to a large set of state-of-the-art Riemannian optimization algorithms.