标题： 强化学习与贝叶斯优化在工业过程智能控制中的交叉：一种基于安全MPC的多目标BO的DPG 显示英文标题

标题： Intersection of Reinforcement Learning and Bayesian Optimization for Intelligent Control of Industrial Processes: A Safe MPC-based DPG using Multi-Objective BO

摘要： 基于模型预测控制（MPC）的强化学习（RL）为深度神经网络（DNN）的RL方法提供了一种结构化且可解释的替代方案，具有较低的计算复杂度和更高的透明度。 然而，标准的MPC-RL方法通常存在收敛速度慢、由于参数化有限而导致的次优策略学习以及在线适应过程中的安全问题。 为了解决这些挑战，我们提出了一种新的框架，将MPC-RL与多目标贝叶斯优化（MOBO）相结合。 所提出的MPC-RL-MOBO利用通过兼容确定性策略梯度（CDPG）方法估计的RL阶段成本及其梯度的噪声评估，并通过期望超体积改进（EHVI）获取函数将它们纳入MOBO算法中。 这种融合使得在模型不完善的情况下，能够高效且安全地调整MPC参数，以实现改进的闭环性能。 一个数值示例展示了所提出方法在控制系统的样本高效、稳定和高性能学习方面的有效性。 显示英文摘要

摘要： Model Predictive Control (MPC)-based Reinforcement Learning (RL) offers a structured and interpretable alternative to Deep Neural Network (DNN)-based RL methods, with lower computational complexity and greater transparency. However, standard MPC-RL approaches often suffer from slow convergence, suboptimal policy learning due to limited parameterization, and safety issues during online adaptation. To address these challenges, we propose a novel framework that integrates MPC-RL with Multi-Objective Bayesian Optimization (MOBO). The proposed MPC-RL-MOBO utilizes noisy evaluations of the RL stage cost and its gradient, estimated via a Compatible Deterministic Policy Gradient (CDPG) approach, and incorporates them into a MOBO algorithm using the Expected Hypervolume Improvement (EHVI) acquisition function. This fusion enables efficient and safe tuning of the MPC parameters to achieve improved closed-loop performance, even under model imperfections. A numerical example demonstrates the effectiveness of the proposed approach in achieving sample-efficient, stable, and high-performance learning for control systems.