The last advances in sequence modeling are mainly based on deep learning approaches. The current state of the art involves the use of variations of the standard LSTM architecture, combined with several tricks that improve the final prediction rates of the trained neural networks. However, in some cases, these adaptations might be too much tuned to the particular problems being addressed. In this article, we show that a very simple idea, to add a direct connection between the input and the output, skipping the recurrent module, leads to an increase of the prediction accuracy in sequence modeling problems related to natural language processing. Experiments carried out on different problems show that the addition of this kind of connection to a recurrent network always improves the results, regardless of the architecture and training-specific details. When this idea is introduced into the models that lead the field, the resulting networks achieve a new state-of-the-art perplexity in language modeling problems.

翻译：序列建模方面的最新进展主要基于深层次的学习方法。目前的最新进展是使用标准 LSTM 结构的变异,加上提高受过训练的神经网络最终预测率的若干技巧。然而,在某些情况下,这些调整可能过于适应正在处理的特殊问题。在本篇文章中,我们表明,为了在输入和输出之间增加直接关联,一个非常简单的想法,跳过经常性模块,导致在与自然语言处理有关的序列建模问题中提高预测准确性。在不同的问题上进行的实验表明,将这种连接添加到经常网络总是能够改善结果,而不管其结构和具体培训细节如何。当这一想法被引入领导这个领域的模型时,由此产生的网络在语言建模问题上产生了新的、最先进的混乱。