Accurate segmentation of neural structures in Electron Microscopy (EM) images is paramount for neuroscience. However, this task is challenged by intricate morphologies, low signal-to-noise ratios, and scarce annotations, limiting the accuracy and generalization of existing methods. To address these challenges, we seek to leverage the priors learned by visual foundation models on a vast amount of natural images to better tackle this task. Specifically, we propose a novel framework that can effectively transfer knowledge from Segment Anything 2 (SAM2), which is pre-trained on natural images, to the EM domain. We first use SAM2 to extract powerful, general-purpose features. To bridge the domain gap, we introduce a Feature-Guided Attention module that leverages semantic cues from SAM2 to guide a lightweight encoder, the Fine-Grained Encoder (FGE), in focusing on these challenging regions. Finally, a dual-affinity decoder generates both coarse and refined affinity maps. Experimental results demonstrate that our method achieves performance comparable to state-of-the-art (SOTA) approaches with the SAM2 weights frozen. Upon further fine-tuning on EM data, our method significantly outperforms existing SOTA methods. This study validates that transferring representations pre-trained on natural images, when combined with targeted domain-adaptive guidance, can effectively address the specific challenges in neuron segmentation.

翻译：电子显微镜（EM）图像中神经结构的精确分割对神经科学至关重要。然而，该任务面临复杂形态、低信噪比和标注稀缺等挑战，限制了现有方法的准确性和泛化能力。为应对这些挑战，我们尝试利用视觉基础模型在大量自然图像上学习到的先验知识来更好地处理此任务。具体而言，我们提出了一种新颖框架，能够有效将基于自然图像预训练的Segment Anything 2（SAM2）的知识迁移至EM领域。我们首先利用SAM2提取强大的通用特征。为弥合领域差异，我们引入了特征引导注意力模块，该模块利用SAM2的语义线索引导轻量级编码器——细粒度编码器（FGE）聚焦于这些具有挑战性的区域。最后，双亲和力解码器同时生成粗粒度与精细化亲和力图。实验结果表明，在SAM2权重冻结的情况下，我们的方法取得了与最先进（SOTA）方法相当的性能。进一步在EM数据上进行微调后，我们的方法显著超越了现有SOTA方法。本研究验证了结合针对性领域自适应指导，迁移自然图像预训练表征能够有效解决神经元分割中的特定挑战。