Adaptive defenses, which optimize at test time, promise to improve adversarial robustness. We categorize such adaptive test-time defenses, explain their potential benefits and drawbacks, and evaluate a representative variety of the latest adaptive defenses for image classification. Unfortunately, none significantly improve upon static defenses when subjected to our careful case study evaluation. Some even weaken the underlying static model while simultaneously increasing inference computation. While these results are disappointing, we still believe that adaptive test-time defenses are a promising avenue of research and, as such, we provide recommendations for their thorough evaluation. We extend the checklist of Carlini et al. (2019) by providing concrete steps specific to adaptive defenses.

翻译：适应性防御在测试时最优化,它有望提高对抗性强力。 我们对这种适应性测试时防御进行分类,解释其潜在好处和缺点,并评估有代表性的最新适应性防御,以进行图像分类。 不幸的是,在接受我们仔细的案例研究评估时,静态防御没有显著改善。有些甚至削弱基本静态模型,同时增加推论计算。 虽然这些结果令人失望,但我们仍然认为适应性测试时防御是一种有希望的研究途径,因此,我们建议对其进行彻底评估。我们通过提供适应性防御的具体步骤,扩大了卡利尼等人(2019年)的清单。