We propose a modular architecture of language-specific encoder-decoders that constitutes a multilingual machine translation system that can be incrementally extended to new languages without the need for retraining the existing system when adding new languages. Differently from previous works, we simultaneously train $N$ languages in all translation directions by alternately freezing encoder or decoder modules, which indirectly forces the system to train in a common intermediate representation for all languages. Experimental results from multilingual machine translation show that we can successfully train this modular architecture improving on the initial languages while falling slightly behind when adding new languages or doing zero-shot translation. Additional comparison of the quality of sentence representation in the task of natural language inference shows that the alternately freezing training is also beneficial in this direction.

翻译：我们提出了一种语言特定编码器-解码器的模块化架构，该架构构成了一个多语言机器翻译系统，能够在不重新训练现有系统的情况下，逐步扩展至新语言。与先前研究不同，我们通过交替冻结编码器或解码器模块，同时训练$N$种语言的所有翻译方向，这间接迫使系统为所有语言训练一个共同的中间表示。多语言机器翻译的实验结果表明，我们能够成功训练此模块化架构，在初始语言上取得改进，但在添加新语言或进行零样本翻译时略有不足。在自然语言推理任务中对句子表示质量的进一步比较表明，交替冻结训练在此方向上也具有益处。