标题： 用于学习超参数的分层优化：在使用高斯过程求解偏微分方程和反问题中的应用 显示英文标题

标题： Bilevel optimization for learning hyperparameters: Application to solving PDEs and inverse problems with Gaussian processes

摘要： 用于解决科学计算和推断问题的方法，如基于核函数和神经网络的偏微分方程（PDEs）方法、反问题和监督学习任务，关键取决于超参数的选择。特别是，这些方法的效果，尤其是它们的准确性、稳定性和泛化能力，强烈依赖于超参数的选择。虽然双层优化为超参数调整提供了一个合理的框架，但其嵌套优化结构在计算上可能非常耗时，尤其是在PDE约束的情况下。在本文中，我们通过采用内层优化步骤的高斯-牛顿线性化，在双层框架内提出了一种高效的超参数优化策略。我们的方法提供了显式更新，消除了重复进行昂贵的PDE求解的需要。因此，外层循环的每次迭代减少为一次线性化的PDE求解，随后进行显式的基于梯度的超参数更新。我们通过将高斯过程模型应用于非线性PDE和PDE反问题来展示所提方法的有效性。大量的数值实验表明，与传统的随机超参数初始化相比，准确性与鲁棒性有显著提升。特别是，使用加法核和神经网络参数化的深度核的实验展示了该方法在高维超参数优化中的可扩展性和有效性。 显示英文摘要

摘要： Methods for solving scientific computing and inference problems, such as kernel- and neural network-based approaches for partial differential equations (PDEs), inverse problems, and supervised learning tasks, depend crucially on the choice of hyperparameters. Specifically, the efficacy of such methods, and in particular their accuracy, stability, and generalization properties, strongly depends on the choice of hyperparameters. While bilevel optimization offers a principled framework for hyperparameter tuning, its nested optimization structure can be computationally demanding, especially in PDE-constrained contexts. In this paper, we propose an efficient strategy for hyperparameter optimization within the bilevel framework by employing a Gauss-Newton linearization of the inner optimization step. Our approach provides closed-form updates, eliminating the need for repeated costly PDE solves. As a result, each iteration of the outer loop reduces to a single linearized PDE solve, followed by explicit gradient-based hyperparameter updates. We demonstrate the effectiveness of the proposed method through Gaussian process models applied to nonlinear PDEs and to PDE inverse problems. Extensive numerical experiments highlight substantial improvements in accuracy and robustness compared to conventional random hyperparameter initialization. In particular, experiments with additive kernels and neural network-parameterized deep kernels demonstrate the method's scalability and effectiveness for high-dimensional hyperparameter optimization.