We present a new zero-shot dense retrieval (ZeroDR) method, COCO-DR, to improve the generalization ability of dense retrieval by combating the distribution shifts between source training tasks and target scenarios. To mitigate the impact of document differences, COCO-DR continues pretraining the language model on the target corpora to adapt the model to target distributions via COtinuous COtrastive learning. To prepare for unseen target queries, COCO-DR leverages implicit Distributionally Robust Optimization (iDRO) to reweight samples from different source query clusters for improving model robustness over rare queries during fine-tuning. COCO-DR achieves superior average performance on BEIR, the zero-shot retrieval benchmark. At BERT Base scale, COCO-DR Base outperforms other ZeroDR models with 60x larger size. At BERT Large scale, COCO-DR Large outperforms the giant GPT-3 embedding model which has 500x more parameters. Our analysis show the correlation between COCO-DR's effectiveness in combating distribution shifts and improving zero-shot accuracy. Our code and model can be found at \url{https://github.com/OpenMatch/COCO-DR}.

翻译：为减少文件差异的影响,COCO-DR继续在目标公司对语言模型进行先期培训,以便通过COTRECO-DR通过COTRA学习使模型适应目标分布。为了准备接受看不见的目标查询,COCO-DR利用不同来源查询群的隐性分布式优化(iDRO)来重新加权样本,以便在微调期间改进对稀有查询的模型稳健性。COCO-DR在BEIR(零光检索基准)上实现了优优优优优平均性能。在BERT基地,COCO-DR基地比其他ZERDR模型大60x大。在BERT大尺度上,COCO-DR大型比GPT-3嵌入模型大,该模型有500x更多的参数。我们的分析显示COCO-DR在打击分发转移和改进零光精确度方面的有效性。我们的代码和模型可以在ORCMM/OUBURM*。我们的代码和模型可以在OBARM/OBARGM*/OGRGRGRGRM*/OGRG/OGRGM}