标题： 使用虚拟对抗训练的分布平滑 显示英文标题

标题： Distributional Smoothing with Virtual Adversarial Training

摘要： 我们提出了局部分布光滑性（LDS），这是一种用于统计模型的新光滑性概念，可以用作正则化项以促进模型分布的光滑性。 我们将基于LDS的正则化命名为虚拟对抗训练（VAT）。 模型在输入数据点处的LDS被定义为其分布针对数据点周围局部扰动的基于KL散度的鲁棒性。 VAT类似于对抗性训练，但区别在于它仅从模型分布本身确定对抗方向而不使用标签信息，使其适用于半监督学习。 VAT的计算成本相对较低。 对于神经网络，可以通过不超过三对前向和反向传播来计算LDS的近似梯度。 当我们将其技术应用于MNIST数据集的有监督和半监督学习时，它在所有训练方法中表现最佳，仅次于当前最先进的基于高度先进的生成模型的方法。 我们还将其方法应用于SVHN和NORB，并确认了我们方法在这两个数据集上优于当前最先进的半监督方法的表现。 显示英文摘要

摘要： We propose local distributional smoothness (LDS), a new notion of smoothness for statistical model that can be used as a regularization term to promote the smoothness of the model distribution. We named the LDS based regularization as virtual adversarial training (VAT). The LDS of a model at an input datapoint is defined as the KL-divergence based robustness of the model distribution against local perturbation around the datapoint. VAT resembles adversarial training, but distinguishes itself in that it determines the adversarial direction from the model distribution alone without using the label information, making it applicable to semi-supervised learning. The computational cost for VAT is relatively low. For neural network, the approximated gradient of the LDS can be computed with no more than three pairs of forward and back propagations. When we applied our technique to supervised and semi-supervised learning for the MNIST dataset, it outperformed all the training methods other than the current state of the art method, which is based on a highly advanced generative model. We also applied our method to SVHN and NORB, and confirmed our method's superior performance over the current state of the art semi-supervised method applied to these datasets.