Unveiling visual semantics from neural signals such as EEG, MEG, and fMRI remains a fundamental challenge due to subject variability and the entangled nature of visual features. Existing approaches primarily align neural activity directly with visual embeddings, but visual-only representations often fail to capture latent semantic dimensions, limiting interpretability and deep robustness. To address these limitations, we propose Bratrix, the first end-to-end framework to achieve multimodal Language-Anchored Vision-Brain alignment. Bratrix decouples visual stimuli into hierarchical visual and linguistic semantic components, and projects both visual and brain representations into a shared latent space, enabling the formation of aligned visual-language and brain-language embeddings. To emulate human-like perceptual reliability and handle noisy neural signals, Bratrix incorporates a novel uncertainty perception module that applies uncertainty-aware weighting during alignment. By leveraging learnable language-anchored semantic matrices to enhance cross-modal correlations and employing a two-stage training strategy of single-modality pretraining followed by multimodal fine-tuning, Bratrix-M improves alignment precision. Extensive experiments on EEG, MEG, and fMRI benchmarks demonstrate that Bratrix improves retrieval, reconstruction, and captioning performance compared to state-of-the-art methods, specifically surpassing 14.3% in 200-way EEG retrieval task. Code and model are available.

翻译：从脑电图（EEG）、脑磁图（MEG）和功能磁共振成像（fMRI）等神经信号中揭示视觉语义，由于受试者间的变异性及视觉特征的纠缠性，仍是一个根本性挑战。现有方法主要将神经活动直接与视觉嵌入对齐，但仅依赖视觉的表征往往难以捕捉潜在的语义维度，限制了可解释性与深层鲁棒性。为应对这些局限，我们提出了Bratrix，这是首个实现多模态语言锚定视觉-大脑对齐的端到端框架。Bratrix将视觉刺激解耦为层次化的视觉与语言语义成分，并将视觉与大脑表征共同投影到一个共享的潜在空间中，从而形成对齐的视觉-语言和大脑-语言嵌入。为模拟类人的感知可靠性并处理噪声神经信号，Bratrix引入了一个新颖的不确定性感知模块，在对齐过程中应用不确定性感知加权。通过利用可学习的语言锚定语义矩阵增强跨模态相关性，并采用单模态预训练后接多模态微调的两阶段训练策略，Bratrix-M提升了对齐精度。在EEG、MEG和fMRI基准数据集上的大量实验表明，Bratrix在检索、重建和字幕生成任务上均优于现有先进方法，尤其在200类EEG检索任务中性能提升超过14.3%。代码与模型已公开。