标题： 将瞬时记忆整合到强化学习智能体中使用水库采样 显示英文标题

标题： Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling

摘要： 情景记忆是一个心理学术语，指的是回忆过去特定事件的能力。 我们认为这种特定类型的记忆的一个优势是，当回忆的信息被发现是有用的时，能够轻松地将信用分配给特定的状态。 受这一想法的启发，以及外部记忆机制在深度学习系统中处理长期依赖关系日益流行的影响，我们提出了一种新算法，该算法使用水库采样过程来维护一个由固定数量的过去状态组成的外部记忆。 该算法使深度强化学习代理能够在线学习，优先记住那些后来被发现有用的状态。 关键的是，这种方法允许对外部记忆的写入过程进行高效的在线梯度估计。 因此，与大多数之前的外部记忆机制不同，它在在线强化学习设置中是可行的。 显示英文摘要

摘要： Episodic memory is a psychology term which refers to the ability to recall specific events from the past. We suggest one advantage of this particular type of memory is the ability to easily assign credit to a specific state when remembered information is found to be useful. Inspired by this idea, and the increasing popularity of external memory mechanisms to handle long-term dependencies in deep learning systems, we propose a novel algorithm which uses a reservoir sampling procedure to maintain an external memory consisting of a fixed number of past states. The algorithm allows a deep reinforcement learning agent to learn online to preferentially remember those states which are found to be useful to recall later on. Critically this method allows for efficient online computation of gradient estimates with respect to the write process of the external memory. Thus unlike most prior mechanisms for external memory it is feasible to use in an online reinforcement learning setting.