标题： 自回归外生系统的有限时间系统辨识与自适应控制 显示英文标题

标题： Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems

摘要： 自回归外生（ARX）系统是用于建模随机线性动态系统（LDS）的输入输出动态系统的通用类别，包括部分可观测的LDS，如LQG系统。 在本工作中，我们研究了未知ARX系统的系统辨识和自适应控制问题。 我们提供了在开环和闭环数据收集下ARX系统的有限时间学习保证。 利用这些保证，我们设计了具有任意强凸或凸二次调节成本的未知ARX系统的自适应控制算法。 在强凸成本函数下，我们设计了一种基于在线梯度下降的自适应控制算法，通过凸控制器重参数化来设计和更新控制器。 我们证明了我们的算法通过探索与承诺方法具有$\tilde{\mathcal{O}}(\sqrt{T})$的遗憾，并且如果使用闭环数据收集在时期内更新模型估计，则在$T$个时间步之后达到最优的遗憾$\text{polylog}(T)$。 对于凸二次成本函数的情况，我们提出了一种自适应控制算法，该算法利用面对不确定性时的乐观原则来设计控制器。 在这种情况下，我们证明探索与承诺方法具有$\tilde{\mathcal{O}}(T^{2/3})$的遗憾上界，并且具有连续模型估计更新的自适应控制在$T$个时间步后达到$\tilde{\mathcal{O}}(\sqrt{T})$的遗憾。 显示英文摘要

摘要： Autoregressive exogenous (ARX) systems are the general class of input-output dynamical systems used for modeling stochastic linear dynamical systems (LDS) including partially observable LDS such as LQG systems. In this work, we study the problem of system identification and adaptive control of unknown ARX systems. We provide finite-time learning guarantees for the ARX systems under both open-loop and closed-loop data collection. Using these guarantees, we design adaptive control algorithms for unknown ARX systems with arbitrary strongly convex or convex quadratic regulating costs. Under strongly convex cost functions, we design an adaptive control algorithm based on online gradient descent to design and update the controllers that are constructed via a convex controller reparametrization. We show that our algorithm has $\tilde{\mathcal{O}}(\sqrt{T})$ regret via explore and commit approach and if the model estimates are updated in epochs using closed-loop data collection, it attains the optimal regret of $\text{polylog}(T)$ after $T$ time-steps of interaction. For the case of convex quadratic cost functions, we propose an adaptive control algorithm that deploys the optimism in the face of uncertainty principle to design the controller. In this setting, we show that the explore and commit approach has a regret upper bound of $\tilde{\mathcal{O}}(T^{2/3})$, and the adaptive control with continuous model estimate updates attains $\tilde{\mathcal{O}}(\sqrt{T})$ regret after $T$ time-steps.