标题： 算法和滚动采样近似策略迭代的界限 显示英文标题

标题： Algorithms and Bounds for Rollout Sampling Approximate Policy Iteration

摘要： 几种无需价值函数的近似策略迭代方案最近被提出，这些方案专注于使用分类器进行策略表示，并将策略学习作为监督学习问题来处理。 使用这些方法找到好的策略不仅需要适当的分类器，还需要覆盖状态空间的可靠最佳动作示例。 到目前为止，关于适当的覆盖方案以及减少此类方法样本复杂性的方法的研究很少，尤其是在连续状态空间中。 本文关注最简单的覆盖方案（状态空间上的离散化网格），并在最简单的（并且之前常用）滚动采样分配策略与一种几乎同样简单的方法之间进行样本复杂性比较。该滚动采样分配策略在每个考虑的状态下平均分配样本，而另一种方法仅在需要时分配样本，且所需的样本显著更少。 显示英文摘要

摘要： Several approximate policy iteration schemes without value functions, which focus on policy representation using classifiers and address policy learning as a supervised learning problem, have been proposed recently. Finding good policies with such methods requires not only an appropriate classifier, but also reliable examples of best actions, covering the state space sufficiently. Up to this time, little work has been done on appropriate covering schemes and on methods for reducing the sample complexity of such methods, especially in continuous state spaces. This paper focuses on the simplest possible covering scheme (a discretized grid over the state space) and performs a sample-complexity comparison between the simplest (and previously commonly used) rollout sampling allocation strategy, which allocates samples equally at each state under consideration, and an almost as simple method, which allocates samples only as needed and requires significantly fewer samples.