Transfer learning is an exciting area of Natural Language Processing that has the potential to both improve model performance and increase data efficiency. This study explores the effects of varying quantities of target task training data on sequential transfer learning in the dialog domain. We hypothesize that a model can utilize the information learned from a source task to better learn a target task, thereby reducing the number of target task training samples required. Unintuitively, our data shows that often target task training data size has minimal effect on how sequential transfer learning performs compared to the same model without transfer learning. Our results lead us to believe that this unexpected result could be due to the effects of catastrophic forgetting, motivating further work into methods that prevent such forgetting.

翻译：传输学习是自然语言处理的一个令人振奋的领域,它有可能改善模型性能和提高数据效率。本研究探讨了在对话域内不同数量的目标任务培训数据对相继转移学习的影响。我们假设模型能够利用从源任务中学到的信息更好地学习目标任务,从而减少所需的目标任务培训样本数量。我们的数据不自然地表明,目标任务培训数据的规模往往对相继转移学习与不转移学习的同一模式相比产生最小影响。我们的结果使我们相信,这一意外结果可能是灾难性的遗忘的影响,从而推动进一步开展工作,制定防止这种遗忘的方法。