标题： 考虑允许性松弛的学习价值对齐策略的算法 显示英文标题

标题： Algorithms for learning value-aligned policies considering admissibility relaxation

摘要： 新兴的\emph{价值感知工程}领域认为，软件代理和系统应具备价值意识，即它们必须根据人类价值观做出决策。在这种情况下，这些代理必须能够明确地推理不同行动方案与这些价值观的契合程度。为此，通常将价值观建模为状态或行动上的偏好，然后进行聚合以确定与特定价值观最契合的行动序列。最近，也考虑了这一层次上的额外价值可接受性约束。然而，通常需要这些约束的宽松版本，这大大增加了计算价值对齐策略的复杂性。为了获得在考虑可接受性放松的情况下做出价值对齐决策的高效算法，我们提出了使用学习技术，特别是我们使用了约束强化学习算法。在本文中，我们提出了两种算法，$\epsilon\text{-}ADQL$用于基于局部对齐的策略及其扩展$\epsilon\text{-}CADQL$用于一系列决策。我们在干旱场景下的水资源分配问题中验证了它们的效率。 显示英文摘要

摘要： The emerging field of \emph{value awareness engineering} claims that software agents and systems should be value-aware, i.e. they must make decisions in accordance with human values. In this context, such agents must be capable of explicitly reasoning as to how far different courses of action are aligned with these values. For this purpose, values are often modelled as preferences over states or actions, which are then aggregated to determine the sequences of actions that are maximally aligned with a certain value. Recently, additional value admissibility constraints at this level have been considered as well. However, often relaxed versions of these constraints are needed, and this increases considerably the complexity of computing value-aligned policies. To obtain efficient algorithms that make value-aligned decisions considering admissibility relaxation, we propose the use of learning techniques, in particular, we have used constrained reinforcement learning algorithms. In this paper, we present two algorithms, $\epsilon\text{-}ADQL$ for strategies based on local alignment and its extension $\epsilon\text{-}CADQL$ for a sequence of decisions. We have validated their efficiency in a water distribution problem in a drought scenario.