标题： 基于功率约束的策略梯度方法用于LQR 显示英文标题

标题： Power-Constrained Policy Gradient Methods for LQR

摘要： 考虑一个离散时间线性二次调节器（LQR）问题，当系统矩阵未知时使用策略梯度下降法求解。 梯度通过具有平均功率约束的发送器在有限时间范围内通过模拟通信在噪声信道上传输。 这是强化学习和网络控制系统的交叉点的一个简单设置。 我们首先考虑一个通信受限的优化框架，在该框架中，梯度下降用于在噪声梯度传输下优化非凸函数。 我们提供了一个最优功率分配算法，以最小化最终迭代的期望最优性误差的上界，并表明自适应功率分配相比标准梯度下降与均匀功率分布可以带来更好的收敛速度。 然后我们将我们的结果应用于LQR设置。 显示英文摘要

摘要： Consider a discrete-time Linear Quadratic Regulator (LQR) problem solved using policy gradient descent when the system matrices are unknown. The gradient is transmitted across a noisy channel over a finite time horizon using analog communication by a transmitter with an average power constraint. This is a simple setup at the intersection of reinforcement learning and networked control systems. We first consider a communication-constrained optimization framework, where gradient descent is applied to optimize a non-convex function under noisy gradient transmission. We provide an optimal power allocation algorithm that minimizes an upper bound on the expected optimality error at the final iteration and show that adaptive power allocation can lead to better convergence rate as compared to standard gradient descent with uniform power distribution. We then apply our results to the LQR setting.