Recently, there has been an abundance of works on designing Deep Neural Networks (DNNs) that are robust to adversarial examples. In particular, a central question is which features of DNNs influence adversarial robustness and, therefore, can be to used to design robust DNNs. In this work, this problem is studied through the lens of compression which is captured by the low-rank structure of weight matrices. It is first shown that adversarial training tends to promote simultaneously low-rank and sparse structure in the weight matrices of neural networks. This is measured through the notions of effective rank and effective sparsity. In the reverse direction, when the low rank structure is promoted by nuclear norm regularization and combined with sparsity inducing regularizations, neural networks show significantly improved adversarial robustness. The effect of nuclear norm regularization on adversarial robustness is paramount when it is applied to convolutional neural networks. Although still not competing with adversarial training, this result contributes to understanding the key properties of robust classifiers.

翻译：最近,设计深神经网络(Deep Neal Neal Network)(DNNS)的工作数量众多,对对抗性实例来说是强有力的,特别是,一个核心问题是,DNNS的哪些特征影响对抗性强健性,因此可用于设计强大的DNNS。在这项工作中,这个问题是通过压缩的镜头研究的,由低级的重量矩阵结构所捕捉到。首先显示,对抗性培训倾向于同时促进神经网络重量矩阵中的低级和稀薄结构。这通过有效等级和有效宽度的概念来衡量。在相反的方向,当低级结构通过核规范规范的正规化和与宽度相结合促使规范化时,神经网络显示低级结构大大改进了对抗性强健性。核规范对对抗性强健性的影响在适用于革命性神经网络时最为重要。尽管与对抗性培训没有竞争,但这一结果有助于理解强的分类者的关键特性。