标题： 状态与参数在循环神经网络中的时间尺度耦合 显示英文标题

标题： Time-Scale Coupling Between States and Parameters in Recurrent Neural Networks

摘要： 我们研究循环神经网络（RNNs）中的门控机制如何在使用固定全局学习率进行训练时，隐式地诱导自适应学习率行为。 这种效应源于状态空间时间尺度（由门控参数化）与梯度下降过程中参数空间动力学之间的耦合。 通过推导漏电积分器和门控RNN的精确雅可比矩阵，我们得到一个一阶展开，明确说明了常数、标量和多维门控如何重塑梯度传播，调节有效步长，并在参数更新中引入各向异性。 这些发现表明，门控不仅控制信息流，还作为数据驱动的预条件器，在参数空间中适应优化轨迹。 我们进一步将门控与学习率调度、动量以及自适应方法如Adam进行了形式类比。 实证模拟验证了这些观点：在多个序列任务中，我们展示了门控引起的依赖延迟的有效学习率和梯度流的方向性集中，多门控模型的表现与Adam产生的各向异性结构相当或更优。 这些结果表明，优化器驱动和门控驱动的自适应性是互补而非等价的机制。 总体而言，这项工作提供了一个统一的动力系统视角，解释了门控如何将状态演化与参数更新耦合在一起，从而说明了为什么门控架构在实践中能够实现鲁棒的可训练性和稳定性。 显示英文摘要

摘要： We study how gating mechanisms in recurrent neural networks (RNNs) implicitly induce adaptive learning-rate behavior, even when training is carried out with a fixed, global learning rate. This effect arises from the coupling between state-space time scales--parametrized by the gates--and parameter-space dynamics during gradient descent. By deriving exact Jacobians for leaky-integrator and gated RNNs, we obtain a first-order expansion that makes explicit how constant, scalar, and multi-dimensional gates reshape gradient propagation, modulate effective step sizes, and introduce anisotropy in parameter updates. These findings reveal that gates not only control information flow, but also act as data-driven preconditioners that adapt optimization trajectories in parameter space. We further draw formal analogies with learning-rate schedules, momentum, and adaptive methods such as Adam. Empirical simulations corroborate these claims: in several sequence tasks, we show that gates induce lag-dependent effective learning rates and directional concentration of gradient flow, with multi-gate models matching or exceeding the anisotropic structure produced by Adam. These results highlight that optimizer-driven and gate-driven adaptivity are complementary but not equivalent mechanisms. Overall, this work provides a unified dynamical systems perspective on how gating couples state evolution with parameter updates, explaining why gated architectures achieve robust trainability and stability in practice.