标题： 分布平滑与虚拟对抗训练 显示英文标题

标题： Distributional Smoothing with Virtual Adversarial Training

摘要： 模型预测的泛化性能在很大程度上取决于我们推断模型的家族，大多数成功的模型家族都具有某种平滑性，因为大多数现实世界现象具有相似的平滑性。 我们想提出局部分布平滑性（LDS），这是统计模型的一种新的平滑性概念，它符合我们对“平滑分布”的直观概念。 模型在某一点的LDS定义为基于KL散度的模型分布对局部扰动的鲁棒性。 遵循对抗训练的工作，我们将基于LDS的正则化称为虚拟对抗训练（VAT）。 VAT类似于对抗训练，但其不同之处在于它仅从模型分布中确定对抗方向，而不使用标签信息。 因此，该技术甚至适用于半监督学习。 当我们将该技术应用于排列不变MNIST数据集的分类任务时，它超越了所有不使用生成模型和预训练的训练方法。 即使与使用高度先进生成模型的最先进方法相比，VAT也表现良好。 显示英文摘要

摘要： The generalization performance of model-based prediction depends largely on the family from which we infer the model, and the majority of successful families of models possess some property of smoothness because most real world phenomena share similar smoothness properties. We would like to propose Local Distributional Smoothness (LDS), a new notion of smoothness for statistical model which respects our intuitive notion of "smooth distribution". The LDS of a model at a point is defined as the KL-divergence based robustness of the model distribution against local perturbation. Following the work of Adversarial training, we named the LDS based regularization as Virtual Adversarial training (VAT). VAT resembles adversarial training, but distinguishes itself in that it determines the adversarial direction from the model distribution alone without using the label information. The technique is therefore applicable even to semi-supervised learning. When we applied our technique to the classification task of the permutation invariant MNIST dataset, it eclipsed all the training methods that are free of generative models and pre-training. VAT also performed well even in comparison to state of the art method that uses a highly advanced generative model.