标题： 随机蒙特卡罗树搜索中的幂均值估计 显示英文标题

标题： Power Mean Estimation in Stochastic Monte-Carlo Tree_Search

摘要： 蒙特卡洛树搜索（MCTS）是一种广泛用于在线规划的策略，它结合了蒙特卡洛采样和前向树搜索。 其成功依赖于树的上限置信度（UCT）算法，这是多臂老虎机问题中UCB方法的扩展。 然而，由于动作选择中的对数奖励项存在错误，UCT的理论基础不完整，导致开发了固定深度-MCTS，采用多项式探索奖励以平衡探索与利用~\citep{shah2022journal}。 UCT和固定深度-MCTS都存在价值估计偏差：加权和低估了最优值，而最大值估值则高估了它~\citep{coulom2006efficient}。 幂均值估计器提供了一个平衡的解决方案，位于平均值和最大值之间。 Power-UCT~\citep{dam2019generalized}结合了该估计器以获得更准确的价值估计，但其理论分析仍不完整。 本文介绍了随机-Power-UCT，一种使用幂均值估计器并针对随机MDP设计的MCTS算法。 我们分析了其在估计根节点价值方面的多项式收敛性，并表明其具有与固定深度-MCTS相同的收敛率$\mathcal{O}(n^{-1/2})$，其中$n$是访问轨迹的数量，后者是前者的一个特例。 我们的理论结果在各种随机MDP环境中通过实证测试得到了验证。 显示英文摘要

摘要： Monte-Carlo Tree Search (MCTS) is a widely-used strategy for online planning that combines Monte-Carlo sampling with forward tree search. Its success relies on the Upper Confidence bound for Trees (UCT) algorithm, an extension of the UCB method for multi-arm bandits. However, the theoretical foundation of UCT is incomplete due to an error in the logarithmic bonus term for action selection, leading to the development of Fixed-Depth-MCTS with a polynomial exploration bonus to balance exploration and exploitation~\citep{shah2022journal}. Both UCT and Fixed-Depth-MCTS suffer from biased value estimation: the weighted sum underestimates the optimal value, while the maximum valuation overestimates it~\citep{coulom2006efficient}. The power mean estimator offers a balanced solution, lying between the average and maximum values. Power-UCT~\citep{dam2019generalized} incorporates this estimator for more accurate value estimates but its theoretical analysis remains incomplete. This paper introduces Stochastic-Power-UCT, an MCTS algorithm using the power mean estimator and tailored for stochastic MDPs. We analyze its polynomial convergence in estimating root node values and show that it shares the same convergence rate of $\mathcal{O}(n^{-1/2})$, with $n$ is the number of visited trajectories, as Fixed-Depth-MCTS, with the latter being a special case of the former. Our theoretical results are validated with empirical tests across various stochastic MDP environments.