As unlabeled data carry rich task-relevant information, they are proven useful for few-shot learning of language model. The question is how to effectively make use of such data. In this work, we revisit the self-training technique for language model fine-tuning and present a state-of-the-art prompt-based few-shot learner, SFLM. Given two views of a text sample via weak and strong augmentation techniques, SFLM generates a pseudo label on the weakly augmented version. Then, the model predicts the same pseudo label when fine-tuned with the strongly augmented version. This simple approach is shown to outperform other state-of-the-art supervised and semi-supervised counterparts on six sentence classification and six sentence-pair classification benchmarking tasks. In addition, SFLM only relies on a few in-domain unlabeled data. We conduct a comprehensive analysis to demonstrate the robustness of our proposed approach under various settings, including augmentation techniques, model scale, and few-shot knowledge transfer across tasks.

翻译：由于未贴标签的数据含有与任务相关的丰富信息,事实证明这些数据对几近地学习语言模型很有用。 问题是如何有效地利用这些数据。 在这项工作中,我们重新研究语言模型微调的自我培训技术,并提出一个最先进的速成少发数学习者SFLM。考虑到通过微弱和强力增强技术对文本样本的两种观点,SFLM在变弱增强的版本上生成了一个假标签。然后,模型在与强力增强版本进行微调时预测同样的假标签。这个简单的方法在6个句级分类和6个句级分类基准任务上超过了其他最先进的受监管和半监督的对应方。此外,SFLLM只依靠少数一些内部未加标签的数据。我们进行全面分析,以证明我们提出的方法在各种环境下的稳健性,包括增强技术、模型规模和少量不同任务的知识传输。