Ensemble attacks integrate the outputs of surrogate models with diverse architectures, which can be combined with various gradient-based attacks to improve adversarial transferability. However, previous work shows unsatisfactory attack performance when transferring across heterogeneous model architectures. The main reason is that the gradient update directions of heterogeneous surrogate models differ widely, making it hard to reduce the gradient variance of ensemble models while making the best of individual model. To tackle this challenge, we design a novel ensemble attack, NAMEA, which for the first time integrates the gradients from the non-attention areas of ensemble models into the iterative gradient optimization process. Our design is inspired by the observation that the attention areas of heterogeneous models vary sharply, thus the non-attention areas of ViTs are likely to be the focus of CNNs and vice versa. Therefore, we merge the gradients respectively from the attention and non-attention areas of ensemble models so as to fuse the transfer information of CNNs and ViTs. Specifically, we pioneer a new way of decoupling the gradients of non-attention areas from those of attention areas, while merging gradients by meta-learning. Empirical evaluations on ImageNet dataset indicate that NAMEA outperforms AdaEA and SMER, the state-of-the-art ensemble attacks by an average of 15.0% and 9.6%, respectively. This work is the first attempt to explore the power of ensemble non-attention in boosting cross-architecture transferability, providing new insights into launching ensemble attacks.

翻译：集成攻击通过整合具有多样化架构的替代模型的输出，结合各类基于梯度的攻击方法，以提升对抗样本的可迁移性。然而，先前研究表明，在跨异构模型架构迁移时，攻击性能往往不尽如人意。其主要原因在于异构替代模型的梯度更新方向差异显著，导致在充分利用各模型特性的同时难以有效降低集成模型的梯度方差。为应对这一挑战，我们设计了一种新颖的集成攻击方法NAMEA，首次将集成模型中非注意力区域的梯度融入迭代梯度优化过程。该设计的灵感来源于观察到异构模型的注意力区域差异显著，因此视觉Transformer（ViT）的非注意力区域可能成为卷积神经网络（CNN）的关注焦点，反之亦然。为此，我们分别融合集成模型中注意力区域与非注意力区域的梯度，以整合CNN与ViT的迁移信息。具体而言，我们开创性地提出一种将非注意力区域梯度与注意力区域梯度解耦的新方法，同时通过元学习进行梯度融合。在ImageNet数据集上的实证评估表明，NAMEA在平均攻击成功率上分别优于当前最先进的集成攻击方法AdaEA和SMER达15.0%和9.6%。本研究首次探索了集成非注意力机制在增强跨架构可迁移性方面的潜力，为发起集成攻击提供了新的视角。