标题： 部分可观测参考策略编程：无需数值优化求解POMDPs 显示英文标题

标题： Partially Observable Reference Policy Programming: Solving POMDPs Sans Numerical Optimisation

摘要： 本文提出了部分可观测参考策略编程，一种新颖的随时在线近似POMDP求解器，它在深入采样有意义的未来历史的同时，强制进行渐进的策略更新。 我们为算法的基本方案提供了理论保证，这些保证指出性能损失由采样近似误差的平均值而不是通常的最大值来限定，考虑到在线规划中的采样稀疏性，这是一个关键要求。 在两个具有动态演变环境的大规模问题上的实证评估——包括科西嘉地区的一个直升机紧急情况，需要大约150个规划步骤——证实了理论结果，并表明我们的求解器明显优于当前的在线基准。 显示英文摘要

摘要： This paper proposes Partially Observable Reference Policy Programming, a novel anytime online approximate POMDP solver which samples meaningful future histories very deeply while simultaneously forcing a gradual policy update. We provide theoretical guarantees for the algorithm's underlying scheme which say that the performance loss is bounded by the average of the sampling approximation errors rather than the usual maximum, a crucial requirement given the sampling sparsity of online planning. Empirical evaluations on two large-scale problems with dynamically evolving environments -- including a helicopter emergency scenario in the Corsica region requiring approximately 150 planning steps -- corroborate the theoretical results and indicate that our solver considerably outperforms current online benchmarks.