The sparsely-gated Mixture of Experts (MoE) can magnify a network capacity with a little computational complexity. In this work, we investigate how multi-lingual Automatic Speech Recognition (ASR) networks can be scaled up with a simple routing algorithm in order to achieve better accuracy. More specifically, we apply the sparsely-gated MoE technique to two types of networks: Sequence-to-Sequence Transformer (S2S-T) and Transformer Transducer (T-T). We demonstrate through a set of ASR experiments on multiple language data that the MoE networks can reduce the relative word error rates by 16.5% and 4.7% with the S2S-T and T-T, respectively. Moreover, we thoroughly investigate the effect of the MoE on the T-T architecture in various conditions: streaming mode, non-streaming mode, the use of language ID and the label decoder with the MoE.

翻译：分散式的专家混合(MOE)可以放大网络能力,使其在计算上略为复杂。 在这项工作中,我们调查了多种语言自动语音识别(ASR)网络如何能以简单的路径算法扩大规模,以便实现更好的准确性。更具体地说,我们将分散式的移动技术应用于两类网络:序列到序列变换器(S2S-T)和变换器转换器(T-T)。我们通过一套关于多种语言数据的ASR实验证明,教育部网络可以将S2S-T和T-T的相对字差率分别降低16.5%和4.7%。此外,我们还深入调查了教育部在不同条件下对T-T结构的影响:流模式、非流式模式、语言识别的使用以及与教育部的标签解密器。