Title: Optimization Algorithm Synthesis based on Integral Quadratic Constraints: A Tutorial Show Chinese title

Title: 基于积分二次约束的优化算法综合：教程

Abstract: We expose in a tutorial fashion the mechanisms which underlie the synthesis of optimization algorithms based on dynamic integral quadratic constraints. We reveal how these tools from robust control allow to design accelerated gradient descent algorithms with optimal guaranteed convergence rates by solving small-sized convex semi-definite programs. It is shown that this extends to the design of extremum controllers, with the goal to regulate the output of a general linear closed-loop system to the minimum of an objective function. Numerical experiments illustrate that we can not only recover gradient decent and the triple momentum variant of Nesterov's accelerated first order algorithm, but also automatically synthesize optimal algorithms even if the gradient information is passed through non-trivial dynamics, such as time-delays. Show Chinese abstract

Abstract: 我们以教程的形式展示了基于动态积分二次约束的优化算法综合机制。我们揭示了这些来自鲁棒控制的工具如何通过求解小型凸半定规划问题来设计具有最优保证收敛率的加速梯度下降算法。结果表明，这种方法可以扩展到极值控制器的设计中，目的是将一般线性闭环系统的输出调节为目标函数的最小值。数值实验表明，我们不仅可以恢复梯度下降法和Nesterov加速一阶算法的三倍动量变体，还可以自动合成最优算法，即使梯度信息通过非平凡的动力学（如时滞）传递也是如此。