Title: Cooperative Optimal Output Tracking for Discrete-Time Multiagent Systems: Stabilizing Policy Iteration Frameworks and Analysis Show Chinese title

Title: 离散时间多智能体系统的合作最优输出跟踪：稳定策略迭代框架与分析

Abstract: In this paper, two model-free optimal output tracking frameworks based on policy iteration for discrete-time multi-agent systems are proposed. First, we establish a framework of stabilizing policy iteration that can start from any initial feedback control policy, relaxing the dependence of traditional policy iteration on the initial stabilizing control policy. Then, another efficient and equivalent $Q$-learning policy iteration framework is developed, which is shown to require only less system data to get the same results as the stabilizing policy iteration. Both frameworks obtain stabilizing control policy by iterating the stabilizing virtual closed-loop system step-by-step to the actual closed-loop system. Multiple explicit schemes for the iteration step-size/coefficient are designed and their stability during the above iterations is analyzed. By using the generated closed-loop stabilizing control policy and two frameworks, the optimal feedback control gain is obtained. The approximate solution of the regulator equations is found by model-free iteration, which leads to the optimal feedforward gain. Finally, the cooperative optimal output tracking is realized by a distributed feedforward-feedback controller. The proposed algorithms are validated by simulation. Show Chinese abstract

Abstract: 本文提出了两种基于策略迭代的离散时间多智能体系统的无模型最优输出跟踪框架。 首先，我们建立了一个可以从任何初始反馈控制策略开始的稳定策略迭代框架，放松了传统策略迭代对初始稳定控制策略的依赖。 然后，开发了另一种高效且等效的$Q$-学习策略迭代框架，该框架被证明只需要更少的系统数据就能得到与稳定策略迭代相同的结果。 两个框架通过逐步迭代稳定虚拟闭环系统到实际闭环系统来获得稳定控制策略。 设计了多种迭代步长/系数的显式方案，并分析了上述迭代过程中的稳定性。 通过使用生成的闭环稳定控制策略和两个框架，得到了最优反馈控制增益。 通过无模型迭代找到了调节器方程的近似解，从而得到最优前馈增益。 最后，通过分布式前馈-反馈控制器实现了合作最优输出跟踪。 所提出的算法通过仿真进行了验证。