In this paper, we conjecture that if the permutation invariance of neural networks is taken into account, SGD solutions will likely have no barrier in the linear interpolation between them. Although it is a bold conjecture, we show how extensive empirical attempts fall short of refuting it. We further provide a preliminary theoretical result to support our conjecture. Our conjecture has implications for lottery ticket hypothesis, distributed training, and ensemble methods.

翻译：在本文中,我们推测,如果考虑到神经网络的变异性,SGD解决方案在它们之间的线性内插中很可能没有障碍。尽管这是一个大胆的猜测,但我们可以证明广泛的经验尝试没有能够反驳它。我们还提供了初步的理论结果来支持我们的推测。我们的推测对彩票假设、分布式培训和共同方法都有影响。