标题： 卷积神经网络的信息瓶颈理论 显示英文标题

标题： Information Bottleneck Theory on Convolutional Neural Networks

摘要： 近年来，许多研究尝试打开深度神经网络的黑箱，并提出了各种理论来理解它。 其中，信息瓶颈（IB）理论声称在训练过程中存在两个不同的阶段，即拟合阶段和压缩阶段。 这一说法由于在解释前馈神经网络内部行为方面的成功而引起了广泛关注。 在本文中，我们应用IB理论来理解卷积神经网络（CNNs）的动态行为，并研究基本特征如卷积层宽度、核大小、网络深度、池化层和多全连接层如何影响CNNs的性能。 特别是，通过在MNIST和Fashion-MNIST基准上的系列实验分析，我们证明在所有这些情况下都没有观察到压缩阶段。 这表明CNNs的行为比前馈神经网络要复杂得多。 显示英文摘要

摘要： Recent years, many researches attempt to open the black box of deep neural networks and propose a various of theories to understand it. Among them, Information Bottleneck (IB) theory claims that there are two distinct phases consisting of fitting phase and compression phase in the course of training. This statement attracts many attentions since its success in explaining the inner behavior of feedforward neural networks. In this paper, we employ IB theory to understand the dynamic behavior of convolutional neural networks (CNNs) and investigate how the fundamental features such as convolutional layer width, kernel size, network depth, pooling layers and multi-fully connected layer have impact on the performance of CNNs. In particular, through a series of experimental analysis on benchmark of MNIST and Fashion-MNIST, we demonstrate that the compression phase is not observed in all these cases. This shows us the CNNs have a rather complicated behavior than feedforward neural networks.