Out-of-distribution (OOD) data poses serious challenges in deployed machine learning models as even subtle changes could incur significant performance drops. Being able to estimate a model's performance on test data is important in practice as it indicates when to trust to model's decisions. We present a simple yet effective method to predict a model's performance on an unknown distribution without any addition annotation. Our approach is rooted in the Optimal Transport theory, viewing test samples' output softmax scores from deep neural networks as empirical samples from an unknown distribution. We show that our method, Confidence Optimal Transport (COT), provides robust estimates of a model's performance on a target domain. Despite its simplicity, our method achieves state-of-the-art results on three benchmark datasets and outperforms existing methods by a large margin.

翻译：由于即使是微小的变化也可能造成显著的性能下降,因此在部署的机器学习模型中,外部分配数据构成严重挑战,因为即使是微小的变化也可能带来显著的性能下降。 能够估计模型在测试数据上的性能在实践上是重要的,因为它表明何时可以相信模型的决定。 我们提出了一个简单而有效的方法来预测模型在未知的分布上的性能,而没有附加任何注释。 我们的方法植根于最佳运输理论,将测试样品从深层神经网络中产生的软体积分数作为未知分布的经验样本。 我们表明,我们的方法,即信任最佳运输(COT),提供了对模型在目标领域的性能的可靠估计。 尽管这种方法很简单,但我们在三个基准数据集上取得了最先进的结果,并且大大超越了现有方法。