Label noise poses a serious threat to deep neural networks (DNNs). Employing robust loss function which reconciles fitting ability with robustness is a simple but effective strategy to handle this problem. However, the widely-used static trade-off between these two factors contradicts the dynamic nature of DNNs learning with label noise, leading to inferior performance. Therefore, we propose a dynamics-aware loss (DAL) to solve this problem. Considering that DNNs tend to first learn generalized patterns, then gradually overfit label noise, DAL strengthens the fitting ability initially, then gradually increases the weight of robustness. Moreover, at the later stage, we let DNNs put more emphasis on easy examples which are more likely to be correctly labeled than hard ones and introduce a bootstrapping term to further reduce the negative impact of label noise. Both the detailed theoretical analyses and extensive experimental results demonstrate the superiority of our method.

翻译：标签噪声对深度神经网络（DNN）构成了严重的威胁。使用既能拟合高性能模型，又能抵御噪声的鲁棒损失函数是处理此问题的简单而有效的策略。然而，这两个因素之间的静态折衷与DNNs动态学习的本质相矛盾，导致性能下降。因此，我们提出了一种考虑DNNs动态学习的标签噪声损失函数（DAL）。考虑到DNNs倾向于先学习广义模式，然后逐渐过度拟合噪声，DAL在初始阶段加强拟合性能，然后逐渐增加鲁棒性的权重。此外，到后期，我们让DNNs更加注重易于例子，这些例子更有可能被正确标记，同时引入自举项来进一步降低标签噪声的负面影响。详细的理论分析和广泛的实验结果都证明了我们方法的优越性。