标题： 连续Dropout 显示英文标题

标题： Continuous Dropout

摘要： Dropout 已被证明是一种有效的训练鲁棒深度网络的算法，因为它能够通过避免特征检测器的共适应来防止过拟合。 目前对 Dropout 的解释包括装袋、朴素贝叶斯、正则化和进化中的性选择。 根据人类大脑中神经元的激活模式，在面对不同情况时，神经元的放电率是随机且连续的，而不是当前 Dropout 所采用的二进制方式。 受此现象启发，我们将传统的二值 Dropout 扩展到连续 Dropout。 一方面，连续 Dropout 比传统的二值 Dropout 更接近人类大脑中神经元的激活特性。 另一方面，我们证明了连续 Dropout 具有避免特征检测器共适应的性质，这表明我们可以提取更多的独立特征检测器用于测试阶段的模型平均。 我们将提出的连续 Dropout 引入前馈神经网络，并在 MNIST、CIFAR-10、SVHN、NORB 和 ILSVRC-12 数据集上与二值 Dropout、自适应 Dropout 和 DropConnect 进行全面比较。 彻底的实验表明，我们的方法在防止特征检测器的共适应方面表现更好，并提高了测试性能。 代码可在以下地址获取：https://github.com/jasonustc/caffe-multigpu/tree/dropout。 显示英文摘要

摘要： Dropout has been proven to be an effective algorithm for training robust deep networks because of its ability to prevent overfitting by avoiding the co-adaptation of feature detectors. Current explanations of dropout include bagging, naive Bayes, regularization, and sex in evolution. According to the activation patterns of neurons in the human brain, when faced with different situations, the firing rates of neurons are random and continuous, not binary as current dropout does. Inspired by this phenomenon, we extend the traditional binary dropout to continuous dropout. On the one hand, continuous dropout is considerably closer to the activation characteristics of neurons in the human brain than traditional binary dropout. On the other hand, we demonstrate that continuous dropout has the property of avoiding the co-adaptation of feature detectors, which suggests that we can extract more independent feature detectors for model averaging in the test stage. We introduce the proposed continuous dropout to a feedforward neural network and comprehensively compare it with binary dropout, adaptive dropout, and DropConnect on MNIST, CIFAR-10, SVHN, NORB, and ILSVRC-12. Thorough experiments demonstrate that our method performs better in preventing the co-adaptation of feature detectors and improves test performance. The code is available at: https://github.com/jasonustc/caffe-multigpu/tree/dropout.