标题： RiemannLoRA：一种无歧义LoRA优化的统一黎曼框架 显示英文标题

标题： RiemannLoRA: A Unified Riemannian Framework for Ambiguity-Free LoRA Optimization

摘要： 低秩适应（LoRA）已成为大型语言模型（LLMs）参数高效微调的广泛采用的标准，显著降低了内存和计算需求。 然而，仍存在一些挑战，包括寻找最佳初始化策略或减轻低秩矩阵分解中的过参数化问题。 在本工作中，我们提出了一种新方法，在统一框架内同时解决这两个挑战。 我们的方法将一组固定秩的LoRA矩阵视为一个平滑流形。 考虑到适配器作为该流形上的元素，可以消除过参数化，而沿着流形确定最快损失减少方向则提供了初始化。 特别注意以数值稳定且计算高效的实现方式来实施我们的方法，使用数值线性代算和黎曼优化的最佳实践。 在LLM和扩散模型架构上的实验结果表明，RiemannLoRA在收敛速度和最终性能方面始终优于标准LoRA及其最先进的改进方法。 显示英文摘要

摘要： Low-Rank Adaptation (LoRA) has become a widely adopted standard for parameter-efficient fine-tuning of large language models (LLMs), significantly reducing memory and computational demands. However, challenges remain, including finding optimal initialization strategies or mitigating overparametrization in low-rank matrix factorization. In this work, we propose a novel approach that addresses both of the challenges simultaneously within a unified framework. Our method treats a set of fixed-rank LoRA matrices as a smooth manifold. Considering adapters as elements on this manifold removes overparametrization, while determining the direction of the fastest loss decrease along the manifold provides initialization. Special care is taken to obtain numerically stable and computationally efficient implementation of our method, using best practices from numerical linear algebra and Riemannian optimization. Experimental results on LLM and diffusion model architectures demonstrate that RiemannLoRA consistently improves both convergence speed and final performance over standard LoRA and its state-of-the-art modifications.