Consistency training regularizes a model by enforcing predictions of original and perturbed inputs to be similar. Previous studies have proposed various augmentation methods for the perturbation but are limited in that they are agnostic to the training model. Thus, the perturbed samples may not aid in regularization due to their ease of classification from the model. In this context, we propose an augmentation method of adding a discrete noise that would incur the highest divergence between predictions. This virtual adversarial discrete noise obtained by replacing a small portion of tokens while keeping original semantics as much as possible efficiently pushes a training model's decision boundary. Experimental results show that our proposed method outperforms other consistency training baselines with text editing, paraphrasing, or a continuous noise on semi-supervised text classification tasks and a robustness benchmark

翻译：一致性培训通过对原始和扰动投入进行类似预测,使模型规范化,对原始和扰动投入进行预测,使模型规范化。以前的研究曾提出各种扰动增强方法,但仅限于对培训模式的不可知性,因此,受扰动样品可能无助于正规化,因为从模型分类容易。在这方面,我们提议一种增强方法,增加离散噪音,在预测中产生最大的差异。这种虚拟对抗性离散噪音,通过替换一小部分符号,同时尽可能有效地保持原始语义,从而尽可能有效地推动培训模式的决定界限。实验结果显示,我们拟议的方法优于其他一致性培训基线,与文字编辑、副写或半监督文本分类任务的连续噪音和稳健性基准相比。