标题： 基于效用的基础设施维护优化的深度多目标强化学习 显示英文标题

标题： Deep Multi-Objective Reinforcement Learning for Utility-Based Infrastructural Maintenance Optimization

摘要： 在本文中，我们引入了多目标深度集中式多智能体演员-评论家（MO-DCMAC），这是一种用于基础设施维护优化的多目标强化学习（MORL）方法，该领域传统上由单目标强化学习（RL）方法主导。 以前的单目标RL方法通过奖励塑造将多个目标（如倒塌概率和成本）结合成一个单一的奖励信号。 相比之下，MO-DCMAC可以直接优化多个目标的策略，即使效用函数是非线性的。 我们使用两个效用函数对MO-DCMAC进行了评估，这两个效用函数以倒塌概率和成本作为输入。 第一个效用函数是阈值效用，其中MO-DCMAC应最小化成本，使得倒塌概率永远不会超过阈值。 第二个效用函数基于资产管理者用于评估维护计划的失效模式、影响和关键性分析（FMECA）方法。 我们在多个维护环境中对MO-DCMAC进行了评估，包括基于阿姆斯特丹历史堤坝案例研究的环境。 MO-DCMAC的性能与基于当前用于构建维护计划的启发式规则的多种基于规则的策略进行了比较。 我们的结果表明，MO-DCMAC在各种环境和效用函数下均优于传统的基于规则的策略。 显示英文摘要

摘要： In this paper, we introduce Multi-Objective Deep Centralized Multi-Agent Actor-Critic (MO- DCMAC), a multi-objective reinforcement learning (MORL) method for infrastructural maintenance optimization, an area traditionally dominated by single-objective reinforcement learning (RL) approaches. Previous single-objective RL methods combine multiple objectives, such as probability of collapse and cost, into a singular reward signal through reward-shaping. In contrast, MO-DCMAC can optimize a policy for multiple objectives directly, even when the utility function is non-linear. We evaluated MO-DCMAC using two utility functions, which use probability of collapse and cost as input. The first utility function is the Threshold utility, in which MO-DCMAC should minimize cost so that the probability of collapse is never above the threshold. The second is based on the Failure Mode, Effects, and Criticality Analysis (FMECA) methodology used by asset managers to asses maintenance plans. We evaluated MO-DCMAC, with both utility functions, in multiple maintenance environments, including ones based on a case study of the historical quay walls of Amsterdam. The performance of MO-DCMAC was compared against multiple rule-based policies based on heuristics currently used for constructing maintenance plans. Our results demonstrate that MO-DCMAC outperforms traditional rule-based policies across various environments and utility functions.