Deciphering how visual stimuli are transformed into cortical responses is a fundamental challenge in computational neuroscience. This visual-to-neural mapping is inherently a one-to-many relationship, as identical visual inputs reliably evoke variable hemodynamic responses across trials, contexts, and subjects. However, existing deterministic methods struggle to simultaneously model this biological variability while capturing the underlying functional consistency that encodes stimulus information. To address these limitations, we propose SynBrain, a generative framework that simulates the transformation from visual semantics to neural responses in a probabilistic and biologically interpretable manner. SynBrain introduces two key components: (i) BrainVAE models neural representations as continuous probability distributions via probabilistic learning while maintaining functional consistency through visual semantic constraints; (ii) A Semantic-to-Neural Mapper acts as a semantic transmission pathway, projecting visual semantics into the neural response manifold to facilitate high-fidelity fMRI synthesis. Experimental results demonstrate that SynBrain surpasses state-of-the-art methods in subject-specific visual-to-fMRI encoding performance. Furthermore, SynBrain adapts efficiently to new subjects with few-shot data and synthesizes high-quality fMRI signals that are effective in improving data-limited fMRI-to-image decoding performance. Beyond that, SynBrain reveals functional consistency across trials and subjects, with synthesized signals capturing interpretable patterns shaped by biological neural variability. Our code is available at https://github.com/MichaelMaiii/SynBrain.
翻译:解析视觉刺激如何转化为皮层响应是计算神经科学中的一个基础性挑战。这种视觉到神经的映射本质上是一种一对多的关系,因为相同的视觉输入在不同实验试次、情境和受试者中会可靠地引发可变的血流动力学响应。然而,现有的确定性方法难以在捕捉编码刺激信息的基础功能一致性的同时,对这种生物变异性进行建模。为应对这些局限性,我们提出了SynBrain,一个以概率化且具有生物学可解释性的方式模拟从视觉语义到神经响应转换的生成框架。SynBrain引入了两个关键组件:(i) BrainVAE通过概率学习将神经表征建模为连续概率分布,同时通过视觉语义约束保持功能一致性;(ii) 语义到神经映射器作为语义传输通路,将视觉语义投影到神经响应流形中,以促进高保真的功能磁共振成像合成。实验结果表明,SynBrain在受试者特定的视觉到功能磁共振成像编码性能上超越了现有最先进方法。此外,SynBrain能够高效地适应新受试者的少样本数据,并合成高质量的功能磁共振成像信号,这些信号能有效提升数据有限条件下的功能磁共振成像到图像解码性能。除此之外,SynBrain揭示了跨试次和受试者的功能一致性,其合成信号捕捉到了由生物神经变异性塑造的可解释模式。我们的代码可在 https://github.com/MichaelMaiii/SynBrain 获取。