标题： 在弱部分可忽略性假设下放宽强化学习中的马尔可夫要求 显示英文标题

标题： Relaxing the Markov Requirements on Reinforcement Learning Under Weak Partial Ignorability

摘要： 不完全数据、混杂效应以及马尔可夫性质的违反是强化学习应用中普遍存在的相互关联的问题。 我们引入了“部分可忽略性”的概念，并利用它建立了自适应强化学习的一个新的收敛定理。 这一理论结果放松了传统$Q$-学习中随机过程的马尔可夫假设，采用罗宾斯-蒙罗随机逼近定理的广义形式来证明最优性。 这一结果对强化学习的大多数活跃子领域有明确的下游影响，并且为扩展到因果推断领域提供了清晰的方向。 显示英文摘要

摘要： Incomplete data, confounding effects, and violations of the Markov property are interrelated problems which are ubiquitous in Reinforcement Learning applications. We introduce the concept of ``partial ignorabilty" and leverage it to establish a novel convergence theorem for adaptive Reinforcement Learning. This theoretical result relaxes the Markov assumption on the stochastic process underlying conventional $Q$-learning, deploying a generalized form of the Robbins-Monro stochastic approximation theorem to establish optimality. This result has clear downstream implications for most active subfields of Reinforcement Learning, with clear paths for extension to the field of Causal Inference.