Transformer encoder-decoder models have shown impressive performance in dialogue modeling. However, as Transformers are inefficient in processing long sequences, dialogue history length often needs to be truncated. To address this problem, we propose a new memory-augmented Transformer that is compatible with existing pre-trained encoder-decoder models and enables efficient preservation of history information. It incorporates a separate memory module alongside the pre-trained Transformer to effectively interchange information between the memory states and the current input context. We evaluate our model on three dialogue datasets and two language modeling datasets. Experimental results show that our method has achieved superior efficiency and performance compared to other pre-trained Transformer baselines.

翻译：变换器编码器- 解码器模型在对话模型中表现出令人印象深刻的性能。 但是,由于变换器在处理长序列方面效率低下,对话历史长度往往需要缩短。 为了解决这个问题,我们提议了一个新的内存强化变换器,该变换器与现有的预先训练的编码器- 解码器模型兼容,并能够有效地保存历史信息。它包含一个单独的内存模块,与预先训练的变换器一起,在记忆状态和当前输入环境之间有效交换信息。我们评估了三个对话数据集和两个语言模型的模型。实验结果表明,我们的方法比其他经过训练的变换器基线提高了效率和性能。