Linear probes are widely used to interpret and evaluate neural representations, yet their reliability remains unclear, as probes may appear accurate in some regimes but collapse unpredictably in others. We uncover a spectral mechanism behind this phenomenon and formalize it as the Spectral Identifiability Principle (SIP), a verifiable Fisher-inspired condition for probe stability. When the eigengap separating task-relevant directions is larger than the Fisher estimation error, the estimated subspace concentrates and accuracy remains consistent, whereas closing this gap induces instability in a phase-transition manner. Our analysis connects eigengap geometry, sample size, and misclassification risk through finite-sample reasoning, providing an interpretable diagnostic rather than a loose generalization bound. Controlled synthetic studies, where Fisher quantities are computed exactly, confirm these predictions and show how spectral inspection can anticipate unreliable probes before they distort downstream evaluation.

翻译：线性探针被广泛用于解释和评估神经表示，但其可靠性仍不明确，因为探针在某些机制下可能表现出准确性，而在其他机制下会以不可预测的方式失效。我们揭示了这一现象背后的谱机制，并将其形式化为谱可辨识性原理（SIP），这是一种可验证的、受Fisher启发的探针稳定性条件。当分离任务相关方向的特征值间隙大于Fisher估计误差时，估计的子空间会集中且准确性保持稳定；反之，当该间隙缩小时，会以相变方式引发不稳定性。我们的分析通过有限样本推理将特征值间隙几何、样本量和误分类风险联系起来，提供了一个可解释的诊断方法，而非宽松的泛化界。在可控的合成研究中，Fisher量被精确计算，这些研究证实了上述预测，并展示了谱检查如何在不可靠探针扭曲下游评估之前预见其问题。