标题： 向脉冲变压器中的相对位置编码迈进 显示英文标题

标题： Toward Relative Positional Encoding in Spiking Transformers

摘要： 脉冲神经网络（SNNs）是受生物启发的网络，它们通过离散的脉冲模拟大脑中神经元之间的通信，由于其能效和时间处理能力，在各种任务中具有巨大潜力。 具有自注意力机制的SNNs（脉冲变换器）最近在序列建模和图像分类等各种任务中表现出显著的进步。 然而，整合位置信息——这对于捕捉数据中的序列关系至关重要——在脉冲变换器中仍然是一个挑战。 在本文中，我们引入了一种用于脉冲变换器的相对位置编码（RPE）的近似方法，利用格雷码作为我们方法的基础。 我们提供了该方法在部分捕捉序列任务中的相对位置信息方面的有效性的全面证明。 此外，我们通过将其适应为适合图像块处理的二维形式，扩展了我们的RPE方法。 我们在多个任务上评估了所提出的RPE方法，包括时间序列预测、文本分类和基于块的图像分类。 我们的实验结果表明，RPE的引入显著提升了性能，因为它有效地捕捉了相对位置信息。 显示英文摘要

摘要： Spiking neural networks (SNNs) are bio-inspired networks that model how neurons in the brain communicate through discrete spikes, which have great potential in various tasks due to their energy efficiency and temporal processing capabilities. SNNs with self-attention mechanisms (Spiking Transformers) have recently shown great advancements in various tasks such as sequential modeling and image classifications. However, integrating positional information, which is essential for capturing sequential relationships in data, remains a challenge in Spiking Transformers. In this paper, we introduce an approximate method for relative positional encoding (RPE) in Spiking Transformers, leveraging Gray Code as the foundation for our approach. We provide comprehensive proof of the method's effectiveness in partially capturing relative positional information for sequential tasks. Additionally, we extend our RPE approach by adapting it to a two-dimensional form suitable for image patch processing. We evaluate the proposed RPE methods on several tasks, including time series forecasting, text classification, and patch-based image classification. Our experimental results demonstrate that the incorporation of RPE significantly enhances performance by effectively capturing relative positional information.