标题： 基于后见之明的动量（HGM）优化器：一种自适应学习率的方法 显示英文标题

标题： Hindsight-Guided Momentum (HGM) Optimizer: An Approach to Adaptive Learning Rate

摘要： 我们引入了事后指导动量（Hindsight-Guided Momentum，HGM），这是一种一阶优化算法，它根据最近更新的方向一致性自适应地调整学习率。 传统的自适应方法，如Adam或RMSprop，仅使用梯度的大小来调整学习动态，常常忽略了重要的几何线索。几何线索指的是方向信息，例如当前梯度与过去更新之间的对齐情况，这反映了优化路径的局部曲率和一致性。 HGM通过引入一个事后机制来评估当前梯度与累积动量之间的余弦相似性来解决这个问题。这使得它能够区分一致和冲突的梯度方向，在更新对齐时增加学习率，并在振荡或噪声区域减少学习率。 结果是一个更灵敏的优化器，在损失曲面的平滑区域加速收敛，同时在更尖锐或更不稳定的区域保持稳定性。 尽管增加了这种自适应性，该方法保持了现有优化器的计算和内存效率。通过更智能地响应优化景观的结构，HGM提供了一种简单而有效的改进，特别是在非凸设置中，如深度神经网络训练。 显示英文摘要

摘要： We introduce Hindsight-Guided Momentum (HGM), a first-order optimization algorithm that adaptively scales learning rates based on the directional consistency of recent updates. Traditional adaptive methods, such as Adam or RMSprop , adapt learning dynamics using only the magnitude of gradients, often overlooking important geometric cues.Geometric cues refer to directional information, such as the alignment between current gradients and past updates, which reflects the local curvature and consistency of the optimization path. HGM addresses this by incorporating a hindsight mechanism that evaluates the cosine similarity between the current gradient and accumulated momentum. This allows it to distinguish between coherent and conflicting gradient directions, increasing the learning rate when updates align and reducing it in regions of oscillation or noise. The result is a more responsive optimizer that accelerates convergence in smooth regions of the loss surface while maintaining stability in sharper or more erratic areas. Despite this added adaptability, the method preserves the computational and memory efficiency of existing optimizers.By more intelligently responding to the structure of the optimization landscape, HGM provides a simple yet effective improvement over existing approaches, particularly in non-convex settings like that of deep neural network training.