Multilingual reasoning remains a significant challenge for large language models (LLMs), with performance disproportionately favoring high-resource languages. Drawing inspiration from cognitive neuroscience, which suggests that human reasoning functions largely independently of language processing, we hypothesize that LLMs similarly encode reasoning and language as separable components that can be disentangled to enhance multilingual reasoning. To evaluate this, we perform a causal intervention by ablating language-specific representations at inference time. Experiments on 10 open-weight LLMs spanning 11 typologically diverse languages show that this language-specific ablation consistently boosts multilingual reasoning performance. Layer-wise analyses further confirm that language and reasoning representations can be effectively disentangled throughout the model, yielding improved multilingual reasoning capabilities, while preserving top-layer language features remains essential for maintaining linguistic fidelity. Compared to post-training methods such as supervised fine-tuning or reinforcement learning, our training-free language-reasoning disentanglement achieves comparable or superior results with minimal computational overhead. These findings shed light on the internal mechanisms underlying multilingual reasoning in LLMs and suggest a lightweight and interpretable strategy for improving cross-lingual generalization.


翻译:多语言推理对大语言模型(LLMs)而言仍是一项重大挑战,其性能往往不成比例地偏向高资源语言。受认知神经科学的启发——该领域研究表明人类推理功能在很大程度上独立于语言处理——我们假设LLMs同样将推理和语言编码为可分离的组件,通过解耦可增强多语言推理能力。为验证此假设,我们在推理时通过消融语言特定表征进行因果干预。在涵盖11种类型学多样语言的10个开源权重LLMs上的实验表明,这种语言特定消融能持续提升多语言推理性能。分层分析进一步证实,语言与推理表征可在整个模型中被有效解耦,从而获得改进的多语言推理能力,同时保留顶层语言特征对维持语言保真度至关重要。与监督微调或强化学习等后训练方法相比,我们无需训练的语言-推理解耦方法以极低计算开销实现了相当或更优的结果。这些发现揭示了LLMs多语言推理的内部机制,并为提升跨语言泛化能力提供了一种轻量且可解释的策略。

0
下载
关闭预览

相关内容

LESS 是一个开源的样式语言,受到 Sass 的影响。严格来说,LESS 是一个嵌套的元语言,符合语法规范的 CSS 语句也是符合规范的 Less 代码。
Top
微信扫码咨询专知VIP会员