标题： BrainFLORA：通过多模态神经嵌入揭示大脑概念表示 显示英文标题

标题： BrainFLORA: Uncovering Brain Concept Representation via Multimodal Neural Embeddings

摘要： 理解大脑如何表示视觉信息是神经科学和人工智能领域的一个基本挑战。 尽管基于人工智能的神经数据解码已经为人类视觉系统提供了见解，但由于EEG、MEG和fMRI等多模态神经成像信号固有的时空不对齐，整合这些信号仍然是一个关键障碍。 目前的方法通常单独分析这些模态，限制了对神经表征的整体看法。 在本研究中，我们引入了BrainFLORA，这是一个统一的框架，用于整合跨模态神经成像数据以构建共享的神经表征。 我们的方法利用了增强的多模态大语言模型（MLLMs）以及特定模态的适配器和任务解码器，在联合受试者视觉检索任务中实现了最先进的性能，并具有扩展多任务处理的潜力。 结合神经成像分析方法，我们进一步揭示了视觉概念表征如何在神经模态之间以及与现实世界物体感知对齐。 我们证明了大脑结构化的视觉概念表征表现出对物理世界刺激的隐式映射，从不同模态的神经成像角度连接了神经科学和机器学习。 除了方法上的进步， BrainFLORA为认知神经科学和脑机接口（BCIs）提供了新的启示。 我们的代码可在https://github.com/ncclab-sustech/BrainFLORA获取。 显示英文摘要

摘要： Understanding how the brain represents visual information is a fundamental challenge in neuroscience and artificial intelligence. While AI-driven decoding of neural data has provided insights into the human visual system, integrating multimodal neuroimaging signals, such as EEG, MEG, and fMRI, remains a critical hurdle due to their inherent spatiotemporal misalignment. Current approaches often analyze these modalities in isolation, limiting a holistic view of neural representation. In this study, we introduce BrainFLORA, a unified framework for integrating cross-modal neuroimaging data to construct a shared neural representation. Our approach leverages multimodal large language models (MLLMs) augmented with modality-specific adapters and task decoders, achieving state-of-the-art performance in joint-subject visual retrieval task and has the potential to extend multitasking. Combining neuroimaging analysis methods, we further reveal how visual concept representations align across neural modalities and with real world object perception. We demonstrate that the brain's structured visual concept representations exhibit an implicit mapping to physical-world stimuli, bridging neuroscience and machine learning from different modalities of neural imaging. Beyond methodological advancements, BrainFLORA offers novel implications for cognitive neuroscience and brain-computer interfaces (BCIs). Our code is available at https://github.com/ncclab-sustech/BrainFLORA.