Title: SAM4EM: Efficient memory-based two stage prompt-free segment anything model adapter for complex 3D neuroscience electron microscopy stacks Show Chinese title

Title: SAM4EM：用于复杂3D神经科学电子显微镜堆栈的高效基于内存的两阶段无提示分割任何模型适配器

Abstract: We present SAM4EM, a novel approach for 3D segmentation of complex neural structures in electron microscopy (EM) data by leveraging the Segment Anything Model (SAM) alongside advanced fine-tuning strategies. Our contributions include the development of a prompt-free adapter for SAM using two stage mask decoding to automatically generate prompt embeddings, a dual-stage fine-tuning method based on Low-Rank Adaptation (LoRA) for enhancing segmentation with limited annotated data, and a 3D memory attention mechanism to ensure segmentation consistency across 3D stacks. We further release a unique benchmark dataset for the segmentation of astrocytic processes and synapses. We evaluated our method on challenging neuroscience segmentation benchmarks, specifically targeting mitochondria, glia, and synapses, with significant accuracy improvements over state-of-the-art (SOTA) methods, including recent SAM-based adapters developed for the medical domain and other vision transformer-based approaches. Experimental results indicate that our approach outperforms existing solutions in the segmentation of complex processes like glia and post-synaptic densities. Our code and models are available at https://github.com/Uzshah/SAM4EM. Show Chinese abstract

Abstract: 我们提出SAM4EM，一种新颖的方法，通过利用分割一切模型（SAM）以及先进的微调策略，对电子显微镜（EM）数据中的复杂神经结构进行3D分割。我们的贡献包括使用两阶段掩码解码开发一个无需提示的适配器用于SAM，以自动生成提示嵌入，一种基于低秩适应（LoRA）的双阶段微调方法，以在有限标注数据下增强分割，以及一种3D记忆注意力机制，以确保在3D堆栈中的分割一致性。我们进一步发布了一个独特的基准数据集，用于星形胶质细胞过程和突触的分割。我们在具有挑战性的神经科学分割基准上评估了我们的方法，特别针对线粒体、胶质细胞和突触，与最先进的（SOTA）方法相比，取得了显著的准确率提升，包括为医学领域开发的最近SAM基础适配器和其他基于视觉变压器的方法。实验结果表明，我们的方法在胶质细胞和突触后密度等复杂过程的分割中优于现有解决方案。我们的代码和模型可在https://github.com/Uzshah/SAM4EM获取。