Large natural language models (such as GPT-3 or T5) demonstrate impressive abilities across a range of general NLP tasks. Here, we show that the knowledge embedded in such models provides a useful inductive bias, not just on traditional NLP tasks, but also in the nontraditional task of training a symbolic reasoning engine. We observe that these engines learn quickly and generalize in a natural way that reflects human intuition. For example, training such a system to model block-stacking might naturally generalize to stacking other types of objects because of structure in the real world that has been partially captured by the language describing it. We study several abstract textual reasoning tasks, such as object manipulation and navigation, and demonstrate multiple types of generalization to novel scenarios and the symbols that comprise them. We also demonstrate the surprising utility of \textit{compositional learning}, where a learner dedicated to mastering a complicated task gains an advantage by training on relevant simpler tasks instead of jumping straight to the complicated task.

翻译：大型自然语言模型( 如 GPT-3 或 T5 ) 展示了在一系列一般 NLP 任务中令人印象深刻的能力。 在这里, 我们显示这些模型中所包含的知识提供了有用的感化偏差, 不仅在传统的 NLP 任务上, 而且在培训象征性推理引擎的非传统任务上都是如此。 我们观察到, 这些引擎以反映人类直觉的自然方式迅速学习和普及。 例如, 训练这种模拟块状拆解的系统会自然而然地将其他类型的物体堆叠起来, 因为描述它的语言已经部分地占据了真实世界的结构。 我们研究了一些抽象的文字推理任务, 如物体操纵和导航, 并展示了对新情景和构成它们的一些符号的多种类型的概括性。 我们还展示了 ktextit{ composeal learning} 的惊人的效用, 专门掌握复杂任务的学习者通过相关更简单的任务培训而不是直接跳到复杂的任务中获得优势 。