标题： 神经算子的优化可以从宽度中受益 显示英文标题

标题： Optimization for Neural Operators can Benefit from Width

摘要： 直接学习函数空间之间映射的神经算子，如深度算子网络（DONs）和傅里叶神经算子（FNOs），已经引起了广泛关注。 尽管DONs和FNOs具有通用逼近保证，但目前尚无使用梯度下降（GD）学习此类网络的优化收敛保证。 在本文中，我们通过提出一个基于GD的优化统一框架来解决这个开放问题，并将其应用于建立DONs和FNOs的收敛保证。 特别是，我们证明了这两种神经算子相关的损失满足两个条件——受限强凸性（RSC）和光滑性——这些条件保证由于GD导致其损失值的减少。 值得注意的是，由于各自模型架构的不同，这两个条件分别适用于每个神经算子。 理论的一个重要结论是，更宽的网络应该能为DONs和FNOs带来更好的优化收敛性。 我们展示了在典型算子学习问题上的实验结果，以支持我们的理论结果。 显示英文摘要

摘要： Neural Operators that directly learn mappings between function spaces, such as Deep Operator Networks (DONs) and Fourier Neural Operators (FNOs), have received considerable attention. Despite the universal approximation guarantees for DONs and FNOs, there is currently no optimization convergence guarantee for learning such networks using gradient descent (GD). In this paper, we address this open problem by presenting a unified framework for optimization based on GD and applying it to establish convergence guarantees for both DONs and FNOs. In particular, we show that the losses associated with both of these neural operators satisfy two conditions -- restricted strong convexity (RSC) and smoothness -- that guarantee a decrease on their loss values due to GD. Remarkably, these two conditions are satisfied for each neural operator due to different reasons associated with the architectural differences of the respective models. One takeaway that emerges from the theory is that wider networks should lead to better optimization convergence for both DONs and FNOs. We present empirical results on canonical operator learning problems to support our theoretical results.