Existed pre-training methods either focus on single-modal tasks or multi-modal tasks, and cannot effectively adapt to each other. They can only utilize single-modal data (i.e. text or image) or limited multi-modal data (i.e. image-text pairs). In this work, we propose a unified-modal pre-training architecture, namely UNIMO, which can effectively adapt to both single-modal and multi-modal understanding and generation tasks. Large scale of free text corpus and image collections can be utilized to improve the capability of visual and textual understanding, and cross-modal contrastive learning (CMCL) is leveraged to align the textual and visual information into a unified semantic space over a corpus of image-text pairs. As the non-paired single-modal data is very rich, our model can utilize much larger scale of data to learn more generalizable representations. Moreover, the textual knowledge and visual knowledge can enhance each other in the unified semantic space. The experimental results show that UNIMO significantly improves the performance of several single-modal and multi-modal downstream tasks. Our code and pre-trained models are public at https://github.com/PaddlePaddle/Research/tree/master/NLP/UNIMO

翻译：在这项工作中,我们提出了一个统一的培训前现代结构,即UNIMO,它能够有效地适应单一模式和多模式的理解和生成任务。可以使用大量的免费文本资料和图像收藏来提高视觉和文字理解的能力,并利用交叉模式对比学习(CMCL)来将文本和视觉信息与一组图像文本对应的统一的语义空间相匹配。由于非典型的单一模式数据非常丰富,我们的模式可以使用大得多的数据来学习更概括化的表达方式。此外,文字知识和视觉知识可以在统一的语义空间中加强彼此。实验结果显示,UNIMO大大改进了数个单一模式和多模式/多模式的UNCLMM/MDRMB/MDRML任务。我们的代码可以大大扩大数据规模,以学习更概括化的表述方式。此外,文字知识和视觉知识可以在统一的语义空间中加强彼此。实验结果显示,UNIMOO大大改进了数个单一模式和多模式/MDRAB/MAR/MAL/MAL/MAR/MALDLDL DODODODADODDLDLDDDLDORDORDORDORDLODODODODODODLODODODODODODODLODODODLDODODODODLDODODLDODODODODLDODODODLDLDODODODODODLDLDLODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODODOD