标题： 关于使用核激活函数学习的稳定性和泛化性 显示英文标题

标题： On the Stability and Generalization of Learning with Kernel Activation Functions

摘要： 在这一简报中，我们研究了最近提出的一类非参数激活函数——核激活函数（KAFs）的泛化性质。 KAFs 在学习过程中引入额外的参数，以便在每个神经元的基础上单独调整非线性，通过每个激活值的廉价核展开实现这一点。 虽然这种灵活性的增加在实践中已被证明能带来显著改进，但其泛化能力的理论证明在文献中尚未得到解决。 在这里，我们利用关于通过随机梯度下降（SGD）训练的非凸模型稳定性性质的最新研究。 通过间接证明所考虑模型的两个关键平滑性质，我们证明了当用SGD以有限步数训练时，带有KAFs的神经网络能够很好地泛化。 有趣的是，我们的分析为选择该模型的一个超参数——标量高斯核的带宽——提供了指导。 简短的实验评估验证了这一证明。 显示英文摘要

摘要： In this brief we investigate the generalization properties of a recently-proposed class of non-parametric activation functions, the kernel activation functions (KAFs). KAFs introduce additional parameters in the learning process in order to adapt nonlinearities individually on a per-neuron basis, exploiting a cheap kernel expansion of every activation value. While this increase in flexibility has been shown to provide significant improvements in practice, a theoretical proof for its generalization capability has not been addressed yet in the literature. Here, we leverage recent literature on the stability properties of non-convex models trained via stochastic gradient descent (SGD). By indirectly proving two key smoothness properties of the models under consideration, we prove that neural networks endowed with KAFs generalize well when trained with SGD for a finite number of steps. Interestingly, our analysis provides a guideline for selecting one of the hyper-parameters of the model, the bandwidth of the scalar Gaussian kernel. A short experimental evaluation validates the proof.