标题： 可自我解释的决策：一种以用户为中心的深度强化学习解释系统 显示英文标题

标题： Decisions that Explain Themselves: A User-Centric Deep Reinforcement Learning Explanation System

摘要： 随着像自动驾驶这样的深度强化学习（RL）系统被广泛部署但仍然高度不透明，开发人员经常使用可解释的强化学习（XRL）工具来更好地理解和与深度RL代理进行交互。 然而，之前的XRL工作采用了一种技术中心的研究方法，忽视了RL开发人员对生成的解释的看法。 通过一项初步研究，我们确定了RL从业者使用XRL方法的主要目标，以及四个会扩大现有XRL方法与这些目标之间差距的陷阱。 这些陷阱包括难以访问的推理过程、不一致或难以理解的解释，以及无法推广的解释。 为了填补发现的差距，我们提出了一种基于反事实推理的解释方法，该方法可以发现RL代理推理过程的细节并生成自然语言解释。 围绕这种方法，我们构建了一个交互式XRL系统，用户可以主动探索解释和影响信息。 在一项有14名参与者的用户研究中，我们验证了与基线方法相比，开发人员使用我们的系统识别出20.9%更多的异常行为和RL代理的局限性，并且在自动驾驶任务中，使用我们的系统使最终用户在可操作性测试中的表现提高了25.1%，在StarCraft II微管理任务中提高了16.9%。 显示英文摘要

摘要： With deep reinforcement learning (RL) systems like autonomous driving being wildly deployed but remaining largely opaque, developers frequently use explainable RL (XRL) tools to better understand and work with deep RL agents. However, previous XRL works employ a techno-centric research approach, ignoring how RL developers perceive the generated explanations. Through a pilot study, we identify major goals for RL practitioners to use XRL methods and four pitfalls that widen the gap between existing XRL methods and these goals. The pitfalls include inaccessible reasoning processes, inconsistent or unintelligible explanations, and explanations that cannot be generalized. To fill the discovered gap, we propose a counterfactual-inference-based explanation method that discovers the details of the reasoning process of RL agents and generates natural language explanations. Surrounding this method, we build an interactive XRL system where users can actively explore explanations and influential information. In a user study with 14 participants, we validated that developers identified 20.9% more abnormal behaviors and limitations of RL agents with our system compared to the baseline method, and using our system helped end users improve their performance in actionability tests by 25.1% in an auto-driving task and by 16.9% in a StarCraft II micromanagement task.