标题： 具有许多通道的贝叶斯深度卷积网络是高斯过程 显示英文标题

标题： Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes

摘要： 宽全连接神经网络（FCNs）和高斯过程（GPs）之间存在先前确定的等价性。 这种等价性使得可以计算出从一个完全贝叶斯、无限宽训练的FCN中得到的测试集预测，而无需实例化FCN，而是通过评估相应的GP来实现。 在本工作中，我们推导了多层卷积神经网络（CNNs）与高斯过程之间的类似等价性，无论是带有池化层还是不带池化层的CNNs，并在CIFAR10上实现了没有可训练核的GPs的最先进结果。 我们还引入了一种蒙特卡洛方法来估计给定神经网络架构对应的GP，即使分析形式中的项太多以至于计算不可行。 令人惊讶的是，在没有池化层的情况下，具有权重共享和不具有权重共享的CNN对应的GPs是相同的。 因此，翻译不变性在有限通道CNNs中使用随机梯度下降（SGD）训练时是有益的，但在无限通道极限的贝叶斯处理中保证不会起作用——这是两种范式之间的定性差异，而在FCN情况下并不存在。 我们通过实验确认，在某些情况下，随着通道数的增加，SGD训练的有限CNN的性能接近于相应的GPs，但经过仔细调整后，SGD训练的CNN可以显著优于其对应的GPs，这表明与完全贝叶斯参数估计相比，SGD训练具有优势。 显示英文摘要

摘要： There is a previously identified equivalence between wide fully connected neural networks (FCNs) and Gaussian processes (GPs). This equivalence enables, for instance, test set predictions that would have resulted from a fully Bayesian, infinitely wide trained FCN to be computed without ever instantiating the FCN, but by instead evaluating the corresponding GP. In this work, we derive an analogous equivalence for multi-layer convolutional neural networks (CNNs) both with and without pooling layers, and achieve state of the art results on CIFAR10 for GPs without trainable kernels. We also introduce a Monte Carlo method to estimate the GP corresponding to a given neural network architecture, even in cases where the analytic form has too many terms to be computationally feasible. Surprisingly, in the absence of pooling layers, the GPs corresponding to CNNs with and without weight sharing are identical. As a consequence, translation equivariance, beneficial in finite channel CNNs trained with stochastic gradient descent (SGD), is guaranteed to play no role in the Bayesian treatment of the infinite channel limit - a qualitative difference between the two regimes that is not present in the FCN case. We confirm experimentally, that while in some scenarios the performance of SGD-trained finite CNNs approaches that of the corresponding GPs as the channel count increases, with careful tuning SGD-trained CNNs can significantly outperform their corresponding GPs, suggesting advantages from SGD training compared to fully Bayesian parameter estimation.