标题： 面向对抗训练的实用彩票假设 显示英文标题

标题： Towards Practical Lottery Ticket Hypothesis for Adversarial Training

摘要： 最近的研究提出了彩票假设，表明对于深度神经网络，存在可训练的子网络，在相当的训练步骤下，其性能可以与原模型相媲美或更优。 尽管这一发现具有启发性，但寻找合适的子网络需要迭代训练和剪枝。 产生的高成本限制了彩票假设的应用。 我们证明上述子网络中存在一个子集，在训练过程中收敛速度显著更快，因此可以缓解成本问题。 我们进行了广泛的实验，以表明在特定的超参数设置下（$e.g.$，精心选择的学习率、剪枝比例和模型容量），这种子网络在各种模型结构中都稳定存在。 作为我们研究结果的一个实际应用，我们展示了这种子网络可以帮助减少对抗训练的总时间，这是一种提高鲁棒性的标准方法，在CIFAR-10上最多可以减少49%，以实现最先进的鲁棒性。 显示英文摘要

摘要： Recent research has proposed the lottery ticket hypothesis, suggesting that for a deep neural network, there exist trainable sub-networks performing equally or better than the original model with commensurate training steps. While this discovery is insightful, finding proper sub-networks requires iterative training and pruning. The high cost incurred limits the applications of the lottery ticket hypothesis. We show there exists a subset of the aforementioned sub-networks that converge significantly faster during the training process and thus can mitigate the cost issue. We conduct extensive experiments to show such sub-networks consistently exist across various model structures for a restrictive setting of hyperparameters ($e.g.$, carefully selected learning rate, pruning ratio, and model capacity). As a practical application of our findings, we demonstrate that such sub-networks can help in cutting down the total time of adversarial training, a standard approach to improve robustness, by up to 49\% on CIFAR-10 to achieve the state-of-the-art robustness.