标题： 具有连续动作的强化学习中的不可测量混杂因素 显示英文标题

标题： Reinforcement Learning with Continuous Actions Under Unmeasured Confounding

摘要： 本文解决了在强化学习中存在未测量的混杂因素时，连续动作空间下离线策略学习的挑战。虽然大多数现有研究集中在部分可观察马尔可夫决策过程（POMDP）中的策略评估，并假设动作空间是离散的，我们通过建立一个新的识别结果推进了这一领域，在无限 horizon 框架下实现了给定目标策略的非参数化估计。利用这一识别结果，我们开发了一种最小最大估计器，并引入了一种基于策略梯度的算法来识别最大化估计策略价值的同类最优策略。此外，我们提供了关于由此产生的最优策略的一致性、有限样本误差界和遗憾界的理论结果。大量的模拟实验和使用德国家庭面板数据的真实世界应用证明了我们所提出方法的有效性。 显示英文摘要

摘要： This paper addresses the challenge of offline policy learning in reinforcement learning with continuous action spaces when unmeasured confounders are present. While most existing research focuses on policy evaluation within partially observable Markov decision processes (POMDPs) and assumes discrete action spaces, we advance this field by establishing a novel identification result to enable the nonparametric estimation of policy value for a given target policy under an infinite-horizon framework. Leveraging this identification, we develop a minimax estimator and introduce a policy-gradient-based algorithm to identify the in-class optimal policy that maximizes the estimated policy value. Furthermore, we provide theoretical results regarding the consistency, finite-sample error bound, and regret bound of the resulting optimal policy. Extensive simulations and a real-world application using the German Family Panel data demonstrate the effectiveness of our proposed methodology.