Prompt tuning, or the conditioning of a frozen pretrained language model (PLM) with soft prompts learned from data, has demonstrated impressive performance on a wide range of NLP tasks. However, prompt tuning requires a large training dataset to be effective and is outperformed by finetuning the entire PLM in data-scarce regimes. Previous work \citep{gu-etal-2022-ppt,vu-etal-2022-spot} proposed to transfer soft prompts pretrained on the source domain to the target domain. In this paper, we explore domain adaptation for prompt tuning, a problem setting where unlabeled data from the target domain are available during pretraining. We propose bOosting Prompt TunIng with doMain Adaptation (OPTIMA), which regularizes the decision boundary to be smooth around regions where source and target data distributions are similar. Extensive experiments demonstrate that OPTIMA significantly enhances the transferability and sample-efficiency of prompt tuning compared to strong baselines. Moreover, in few-shot settings, OPTIMA exceeds full-model tuning by a large margin.

翻译：快速调整,或对从数据中学会的软性先导语言模型(PLM)进行调控,显示了大量NLP任务方面的令人印象深刻的业绩;然而,快速调控需要大量的培训数据集才能有效,并且通过在数据分离制度中对整个PLM进行微调而优于整个PLM。先前的工作 \ citep{gu-etal-2022-ppt,vu-etal-2022-spot} 提议将源域预先训练的软性提示器转让到目标域。在本文中,我们探索了用于快速调控的域,这是一个在培训前可得到目标域未加标记的数据的设置问题。我们建议用 domain 适应(OPTIMA) 来规范决策界限,以使源和目标数据分布相类似的区域能够平稳。 广泛的实验表明, ALMIMA 与强的基线相比,大大提高了快速调的可转移性和抽样效率。此外,在几发环境中, OPRIMA超过全模的幅度。