Discovering the logical sequence of events is one of the cornerstones in Natural Language Understanding. One approach to learn the sequence of events is to study the order of sentences in a coherent text. Sentence ordering can be applied in various tasks such as retrieval-based Question Answering, document summarization, storytelling, text generation, and dialogue systems. Furthermore, we can learn to model text coherence by learning how to order a set of shuffled sentences. Previous research has relied on RNN, LSTM, and BiLSTM architecture for learning text language models. However, these networks have performed poorly due to the lack of attention mechanisms. We propose an algorithm for sentence ordering in a corpus of short stories. Our proposed method uses a language model based on Universal Transformers (UT) that captures sentences' dependencies by employing an attention mechanism. Our method improves the previous state-of-the-art in terms of Perfect Match Ratio (PMR) score in the ROCStories dataset, a corpus of nearly 100K short human-made stories. The proposed model includes three components: Sentence Encoder, Language Model, and Sentence Arrangement with Brute Force Search. The first component generates sentence embeddings using SBERT-WK pre-trained model fine-tuned on the ROCStories data. Then a Universal Transformer network generates a sentence-level language model. For decoding, the network generates a candidate sentence as the following sentence of the current sentence. We use cosine similarity as a scoring function to assign scores to the candidate embedding and the embeddings of other sentences in the shuffled set. Then a Brute Force Search is employed to maximize the sum of similarities between pairs of consecutive sentences.

翻译：发现事件的逻辑序列是自然语言理解中的基石之一。 学习事件序列的方法之一是在连贯的文本中研究句子顺序。 判决顺序可以应用到基于检索的问答、 文档总和、 故事叙事、 文本生成和对话系统等各种任务中。 此外, 我们可以学习如何订购一套拼接的句子来学习文本的一致性。 先前的研究依靠 RNN、 LSTM 和 BILSTM 的学习文本语言模型。 然而, 这些网络由于缺乏关注机制而表现不佳。 我们建议了在短故事堆中排序句子的算法。 我们提议的方法使用基于通用变换器(UT) 的语文模式, 通过关注机制来捕捉判决依赖性。 我们的方法改进了在 ROCStories 模型( PMRMRM) 中先前的状态, 这是近100KSDRT 快速读取的句子。 拟议的模式包括三个组成部分: 句子解析模式、 语言模型, 和在类似的 RUR- RUR- RURE 排序中, 将 RURE- Reving the the Reving the Reving the Reving the salding exalding the the she the she the she the she daldalbalbalbaldaldaldaldaldalding the squt the she