标题： 提升机器学习中的能量守恒下降法：理论与实践 显示英文标题

标题： Improving Energy Conserving Descent for Machine Learning: Theory and Practice

摘要： 我们发展了能量守恒下降（ECD）的理论，并引入了ECDSep，这是一种基于梯度的优化算法，能够处理凸优化和非凸优化问题。该方法基于优化作为合适混沌能量守恒动力系统物理演化的新型ECD框架，即使对于没有对称性的通用高维问题，也能实现低损失主导的结果分布的解析控制。与之前对此概念的实现相比，我们利用理论控制来改进动态特性和引发混沌的元素，从而提高性能，同时简化针对不同问题类别的优化算法的超参数调整。我们在广泛的机器学习问题上通过随机梯度下降（SGD）、Adam和AdamW等流行优化方法进行了实证比较，在每个任务中都取得了与其最佳表现相当甚至更优的性能。我们识别出分析中的局限性，指出了进一步改进的可能性。 显示英文摘要

摘要： We develop the theory of Energy Conserving Descent (ECD) and introduce ECDSep, a gradient-based optimization algorithm able to tackle convex and non-convex optimization problems. The method is based on the novel ECD framework of optimization as physical evolution of a suitable chaotic energy-conserving dynamical system, enabling analytic control of the distribution of results - dominated at low loss - even for generic high-dimensional problems with no symmetries. Compared to previous realizations of this idea, we exploit the theoretical control to improve both the dynamics and chaos-inducing elements, enhancing performance while simplifying the hyper-parameter tuning of the optimization algorithm targeted to different classes of problems. We empirically compare with popular optimization methods such as SGD, Adam and AdamW on a wide range of machine learning problems, finding competitive or improved performance compared to the best among them on each task. We identify limitations in our analysis pointing to possibilities for additional improvements.