The robustness of machine learning algorithms to distributions shift is primarily discussed in the context of supervised learning (SL). As such, there is a lack of insight on the robustness of the representations learned from unsupervised methods, such as self-supervised learning (SSL) and auto-encoder based algorithms (AE), to distribution shift. We posit that the input-driven objectives of unsupervised algorithms lead to representations that are more robust to distribution shift than the target-driven objective of SL. We verify this by extensively evaluating the performance of SSL and AE on both synthetic and realistic distribution shift datasets. Following observations that the linear layer used for classification itself can be susceptible to spurious correlations, we evaluate the representations using a linear head trained on a small amount of out-of-distribution (OOD) data, to isolate the robustness of the learned representations from that of the linear head. We also develop "controllable" versions of existing realistic domain generalisation datasets with adjustable degrees of distribution shifts. This allows us to study the robustness of different learning algorithms under versatile yet realistic distribution shift conditions. Our experiments show that representations learned from unsupervised learning algorithms generalise better than SL under a wide variety of extreme as well as realistic distribution shifts.

翻译：机器学习算法对于分销转换的稳健性主要在监督学习(SL)的背景下讨论。因此,对于从自我监督学习(SSL)和基于自动编码算法(AE)等未经监督的方法到分销转换的表达方式的稳健性缺乏洞察力。我们认为,未经监督的算法的输入驱动目标导致比SL的目标驱动的表达方式更稳健地进行分销转移。我们通过广泛评价SSL和AE在合成和现实分销转换数据集方面的性能来核实这一点。在发现用于分类的线性层本身可能容易产生虚假的相关性之后,我们利用受过少量分配外分配数据培训的线性头来评估这些表述方式。我们认为,未经监督的运算法的稳健性与线性头的数据分开。我们还开发了现有现实的域通用数据集的“可控性”版本,可调整的分布变化程度。这使我们得以研究在灵活但现实的分布变化条件下的不同学习算法的稳健性。我们进行了实验,在现实的分布变化中学习了比现实的高度变化的细化分析。