We propose a simple modification to standard ResNet architectures--L2 normalization over feature space--that substantially improves out-of-distribution (OoD) performance on the previously proposed Deep Deterministic Uncertainty (DDU) benchmark. We show that this change also induces early Neural Collapse (NC), an effect linked to better OoD performance. Our method achieves comparable or superior OoD detection scores and classification accuracy in a small fraction of the training time of the benchmark. Additionally, it substantially improves worst case OoD performance over multiple, randomly initialized models. Though we do not suggest that NC is the sole mechanism or a comprehensive explanation for OoD behaviour in deep neural networks (DNN), we believe NC's simple mathematical and geometric structure can provide a framework for analysis of this complex phenomenon in future work.

翻译：我们建议对标准ResNet架构-L2对地物空间的标准化进行简单修改,大大改进了先前提议的深确定性不确定性基准在分配(OoD)方面的绩效。我们表明,这一修改还导致早期神经崩溃,这是与更好的OOD性能相关联的效应。我们的方法在基准培训时间的一小部分时间里达到可比较或优的OOD检测分数和分类精确度。此外,它大大改善了多类随机初始化模型的最坏情况OOD性能。虽然我们不认为NC是深层神经网络中OD行为的唯一机制或全面解释,但我们认为NC的简单数学和几何结构可以为未来工作中分析这一复杂现象提供一个框架。