What have language models (LMs) learned about grammar? This question remains hotly debated, with major ramifications for linguistic theory. However, since probability and grammaticality are distinct notions in linguistics, it is not obvious what string probabilities can reveal about an LM's underlying grammatical knowledge. We present a theoretical analysis of the relationship between grammar, meaning, and string probability, based on simple assumptions about the generative process of corpus data. Our framework makes three predictions, which we validate empirically using 280K sentence pairs in English and Chinese: (1) correlation between the probability of strings within minimal pairs, i.e., string pairs with minimal semantic differences; (2) correlation between models' and humans' deltas within minimal pairs; and (3) poor separation in probability space between unpaired grammatical and ungrammatical strings. Our analyses give theoretical grounding for using probability to learn about LMs' structural knowledge, and suggest directions for future work in LM grammatical evaluation.

翻译：语言模型（LMs）学到了哪些语法知识？这一问题在语言学理论中仍备受争议，具有重要影响。然而，由于概率与语法性在语言学中是不同概念，字符串概率能揭示语言模型底层语法知识的程度并不明确。基于对语料库数据生成过程的简单假设，我们提出了一个关于语法、意义与字符串概率之间关系的理论分析框架。该框架作出三项预测，并通过英语和中文的28万句对进行实证验证：（1）最小对（即语义差异极小的字符串对）内字符串概率的相关性；（2）模型与人类在最小对中概率差异（Δ值）的相关性；（3）未配对合法句与非法句在概率空间中的低区分度。我们的分析为利用概率探究语言模型的结构知识提供了理论依据，并为语言模型语法评估的未来研究方向提供了启示。