In this paper, we present the first detailed analysis of how optimization hyperparameters -- such as learning rate, weight decay, momentum, and batch size -- influence robustness against both transfer-based and query-based attacks. Supported by theory and experiments, our study spans a variety of practical deployment settings, including centralized training, ensemble learning, and distributed training. We uncover a striking dichotomy: for transfer-based attacks, decreasing the learning rate significantly enhances robustness by up to $64\%$. In contrast, for query-based attacks, increasing the learning rate consistently leads to improved robustness by up to $28\%$ across various settings and data distributions. Leveraging these findings, we explore -- for the first time -- the optimization hyperparameter design space to jointly enhance robustness against both transfer-based and query-based attacks. Our results reveal that distributed models benefit the most from hyperparameter tuning, achieving a remarkable tradeoff by simultaneously mitigating both attack types more effectively than other training setups.

翻译：本文首次系统分析了优化超参数——如学习率、权重衰减、动量与批大小——如何影响模型对迁移攻击与查询攻击的鲁棒性。通过理论分析与实验验证，本研究覆盖了集中式训练、集成学习及分布式训练等多种实际部署场景。我们发现一个显著的二分现象：对于迁移攻击，降低学习率可显著提升鲁棒性，最高达$64\\%$；相反，对于查询攻击，提高学习率能在不同场景与数据分布下持续增强鲁棒性，最高提升$28\\%$。基于这些发现，我们首次探索了优化超参数的设计空间，以协同提升模型对两类攻击的鲁棒性。结果表明，分布式模型从超参数调优中获益最大，能实现更有效的权衡，相比其他训练配置更显著地同时缓解两类攻击。