标题： 自助抽样式汤普森采样与深度探索 显示英文标题

标题： Bootstrapped Thompson Sampling and Deep Exploration

摘要： 本文献提出了一种新的方法来实现汤普森采样（Thompson sampling）所达成的探索效果，但无需显式地维护或从后验分布中采样。该方法基于一种自助法（bootstrap technique），利用观测数据和人工生成的数据组合。后者用于推导先验分布，正如我们将展示的那样，这对于有效的探索至关重要。我们解释了这种方法如何应用于多臂老虎机（multi-armed bandit）和强化学习问题，并探讨了它与汤普森采样的关系。由于在这些场景下维护或从后验分布中生成样本在计算上变得不可行，因此这种方法特别适用于探索与深度学习相结合的上下文中。 显示英文摘要

摘要： This technical note presents a new approach to carrying out the kind of exploration achieved by Thompson sampling, but without explicitly maintaining or sampling from posterior distributions. The approach is based on a bootstrap technique that uses a combination of observed and artificially generated data. The latter serves to induce a prior distribution which, as we will demonstrate, is critical to effective exploration. We explain how the approach can be applied to multi-armed bandit and reinforcement learning problems and how it relates to Thompson sampling. The approach is particularly well-suited for contexts in which exploration is coupled with deep learning, since in these settings, maintaining or generating samples from a posterior distribution becomes computationally infeasible.