标题： Robbins-Monro 条件用于持续探索学习策略 显示英文标题

标题： Robbins-Monro conditions for persistent exploration learning strategies

摘要： 我们提出了一些简单的假设，这些假设隐含了针对具有局部学习速率的 $Q$-学习算法的Robbins-Monro条件，局部学习速率取决于特定状态-动作对的访问次数（局部时钟）和迭代次数（全局时钟）。 假设马尔可夫决策过程是通信性的，并且学习策略确保持续探索。 对学习率关于局部时钟和全局时钟的功能依赖性施加了限制。 该结果部分证实了Bradkte（1994）的猜想。 显示英文摘要

摘要： We formulate simple assumptions, implying the Robbins-Monro conditions for the $Q$-learning algorithm with the local learning rate, depending on the number of visits of a particular state-action pair (local clock) and the number of iteration (global clock). It is assumed that the Markov decision process is communicating and the learning policy ensures the persistent exploration. The restrictions are imposed on the functional dependence of the learning rate on the local and global clocks. The result partially confirms the conjecture of Bradkte (1994).