Title: Enhancing Certifiable Semantic Robustness via Robust Pruning of Deep Neural Networks Show Chinese title

Title: 通过深度神经网络的鲁棒剪枝增强可认证语义鲁棒性

Abstract: Deep neural networks have been widely adopted in many vision and robotics applications with visual inputs. It is essential to verify its robustness against semantic transformation perturbations, such as brightness and contrast. However, current certified training and robustness certification methods face the challenge of over-parameterization, which hinders the tightness and scalability due to the over-complicated neural networks. To this end, we first analyze stability and variance of layers and neurons against input perturbation, showing that certifiable robustness can be indicated by a fundamental Unbiased and Smooth Neuron metric (USN). Based on USN, we introduce a novel neural network pruning method that removes neurons with low USN and retains those with high USN, thereby preserving model expressiveness without over-parameterization. To further enhance this pruning process, we propose a new Wasserstein distance loss to ensure that pruned neurons are more concentrated across layers. We validate our approach through extensive experiments on the challenging robust keypoint detection task, which involves realistic brightness and contrast perturbations, demonstrating that our method achieves superior robustness certification performance and efficiency compared to baselines. Show Chinese abstract

Abstract: 深度神经网络已被广泛应用于许多具有视觉输入的视觉和机器人应用中。 验证其对语义变换扰动（如亮度和对比度）的鲁棒性至关重要。 然而，当前的认证训练和鲁棒性认证方法面临参数过多的问题，这由于神经网络过于复杂而阻碍了紧致性和可扩展性。 为此，我们首先分析了层和神经元对输入扰动的稳定性和方差，表明可认证的鲁棒性可以通过一个基本的无偏和平滑神经元指标（USN）来指示。 基于USN，我们引入了一种新的神经网络剪枝方法，移除USN较低的神经元并保留USN较高的神经元，从而在不参数过多的情况下保持模型的表达能力。 为了进一步增强这一剪枝过程，我们提出了一种新的Wasserstein距离损失，以确保剪枝后的神经元在各层中更加集中。 我们通过在具有挑战性的鲁棒关键点检测任务上的大量实验验证了我们的方法，该任务涉及真实的亮度和对比度扰动，结果表明，与基线方法相比，我们的方法在鲁棒性认证性能和效率方面表现更优。