Training deep neural networks may be challenging in real world data. Using models as black-boxes, even with transfer learning, can result in poor generalization or inconclusive results when it comes to small datasets or specific applications. This tutorial covers the basic steps as well as more recent options to improve models, in particular, but not restricted to, supervised learning. It can be particularly useful in datasets that are not as well-prepared as those in challenges, and also under scarce annotation and/or small data. We describe basic procedures: as data preparation, optimization and transfer learning, but also recent architectural choices such as use of transformer modules, alternative convolutional layers, activation functions, wide and deep networks, as well as training procedures including as curriculum, contrastive and self-supervised learning.

翻译：在现实世界数据中,培训深层神经网络可能具有挑战性。使用模型作为黑箱,即使进行传输学习,在小型数据集或具体应用程序方面,也可能造成不全面或无结果的结果。这一指导性内容涵盖基本步骤以及最新的改进模型的备选方案,特别是但不局限于监督学习。它对于没有像挑战中那样做好准备的数据集,以及缺乏说明和(或)小数据的数据集可能特别有用。我们描述了基本程序:例如数据编制、优化和传输学习,以及最近的建筑选择,如变压器模块的使用、替代的交替层、激活功能、宽广而深的网络以及培训程序,包括课程、对比式和自我监督的学习。