Large Language Models (LLMs) have demonstrated impressive abilities in symbol processing through in-context learning (ICL). This success flies in the face of decades of critiques asserting that artificial neural networks cannot master abstract symbol manipulation. We seek to understand the mechanisms that can enable robust symbol processing in transformer networks, illuminating both the unanticipated success, and the significant limitations, of transformers in symbol processing. Borrowing insights from symbolic AI and cognitive science on the power of Production System architectures, we develop a high-level Production System Language, PSL, that allows us to write symbolic programs to do complex, abstract symbol processing, and create compilers that precisely implement PSL programs in transformer networks which are, by construction, 100% mechanistically interpretable. The work is driven by study of a purely abstract (semantics-free) symbolic task that we develop, Templatic Generation (TGT). Although developed through study of TGT, PSL is, we demonstrate, highly general: it is Turing Universal. The new type of transformer architecture that we compile from PSL programs suggests a number of paths for enhancing transformers' capabilities at symbol processing. We note, however, that the work we report addresses computability, and not learnability, by transformer networks. Note: The first section provides an extended synopsis of the entire paper.

翻译：大型语言模型（LLMs）通过上下文学习（ICL）在符号处理方面展现出卓越能力。这一成功与数十年来关于人工神经网络无法掌握抽象符号操作的批判观点形成鲜明对比。本研究旨在探究Transformer网络实现稳健符号处理的机制，以阐明其在符号处理中既取得意外成功又存在显著局限性的原因。借鉴符号人工智能和认知科学中关于产生式系统架构能力的见解，我们开发了一种高级产生式系统语言PSL，该语言允许编写符号程序以执行复杂、抽象的符号处理，并创建编译器将PSL程序精确实现在Transformer网络中——这些网络在构造上具有100%的机制可解释性。本工作通过研究我们设计的纯抽象（无语义）符号任务——模板生成（TGT）来推进。尽管基于TGT研究开发，但PSL被证明具有高度通用性：它是图灵完备的。从PSL程序编译出的新型Transformer架构为增强Transformer的符号处理能力提供了多种路径。然而需要指出，本研究关注的是Transformer网络的可计算性，而非其可学习性。注：第一部分提供了全文的扩展概要。