Many recent successes in sentence representation learning have been achieved by simply fine-tuning on the Natural Language Inference (NLI) datasets with triplet loss or siamese loss. Nevertheless, they share a common weakness: sentences in a contradiction pair are not necessarily from different semantic categories. Therefore, optimizing the semantic entailment and contradiction reasoning objective alone is inadequate to capture the high-level semantic structure. The drawback is compounded by the fact that the vanilla siamese or triplet losses only learn from individual sentence pairs or triplets, which often suffer from bad local optima. In this paper, we propose PairSupCon, an instance discrimination based approach aiming to bridge semantic entailment and contradiction understanding with high-level categorical concept encoding. We evaluate PairSupCon on various downstream tasks that involve understanding sentence semantics at different granularities. We outperform the previous state-of-the-art method with $10\%$--$13\%$ averaged improvement on eight clustering tasks, and $5\%$--$6\%$ averaged improvement on seven semantic textual similarity (STS) tasks.

翻译：近来,通过微调自然语言推断数据集(NLI)的三重损失或局部损失,在量刑代表学习方面取得了许多成功。然而,它们有一个共同的弱点:对矛盾的判刑不一定来自不同的语义类别。因此,仅靠优化语义要求和矛盾推理目标不足以捕捉高层语义结构。香草双胞胎或三重损失仅从个别判刑或三重数据中学习,往往受到当地不良选择的影响,这增加了缺点。在本文件中,我们提议PairSupCon,这是一种基于实例的歧视做法,旨在将语义要求和对高级绝对概念编码的矛盾理解联系起来。我们评估下游任务中涉及理解不同颗粒的语义结构。我们超越了先前的状态-艺术方法,8项组合任务的平均改进额为10 美元- 13 美元,7项语义相似任务的平均改进额为5 美元-6 美元(ST)。