标题： 风险敏感线性二次高斯控制的鲁棒强化学习 显示英文标题

标题： Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Control

摘要： 本文提出了一种针对具有模型失配的离散时间线性系统的新型鲁棒强化学习框架，这种模型失配可能源于仿真到现实的差距。 关键策略之一是运用控制理论中的先进方法。 利用经典的风险敏感线性二次高斯控制的公式化表达，提出了一种双环策略优化算法以生成鲁棒最优控制器。 证明了该双环策略优化算法全局一致收敛，并且在学习过程中对扰动具有鲁棒性。 这一鲁棒性属性被称为小扰动输入状态稳定性，保证只要每个学习步长的扰动相对较小，所提出的策略优化算法就能收敛到最优控制器的一个小邻域内。 此外，在系统动力学未知的情况下，提出了一种新的无模型异策略策略优化算法。 最后，通过数值例子展示了所提出的算法。 显示英文摘要

摘要： This paper proposes a novel robust reinforcement learning framework for discrete-time linear systems with model mismatch that may arise from the sim-to-real gap. A key strategy is to invoke advanced techniques from control theory. Using the formulation of the classical risk-sensitive linear quadratic Gaussian control, a dual-loop policy optimization algorithm is proposed to generate a robust optimal controller. The dual-loop policy optimization algorithm is shown to be globally and uniformly convergent, and robust against disturbances during the learning process. This robustness property is called small-disturbance input-to-state stability and guarantees that the proposed policy optimization algorithm converges to a small neighborhood of the optimal controller as long as the disturbance at each learning step is relatively small. In addition, when the system dynamics is unknown, a novel model-free off-policy policy optimization algorithm is proposed. Finally, numerical examples are provided to illustrate the proposed algorithm.