标题： 图神经网络作为单细胞转录组学中变压器的替代方案 显示英文标题

标题： Graph Neural Networks as a Substitute for Transformers in Single-Cell Transcriptomics

摘要： 图神经网络（GNNs）和Transformer在与感兴趣节点的特征交互的编码策略上具有显著的相似性，其中Transformer使用查询-键分数，而GNNs使用边。 与无法编码相对位置的GNNs相比，Transformer利用动态注意力能力更好地表示相对关系，因此成为大规模序列预训练的标准主干。 然而，这种细微的差异促使我们思考：如果位置不再重要，我们是否可以在某些领域如单细胞转录组学中用图神经网络替代Transformer？ 在本文中，我们首先探讨了GNNs和Transformer之间的相似性和差异性，特别是在相对位置方面。 此外，我们设计了一个合成示例来说明在样本中没有token相对位置时它们的等价性。 最后，我们在一个大规模的与位置无关的数据集——单细胞转录组学上进行了广泛的实验，发现GNNs在消耗更少计算资源的情况下，与Transformer相比表现出具有竞争力的性能。 这些发现为单细胞转录组学领域的研究人员提供了新的见解，挑战了Transformer总是最优选择的普遍观念。 显示英文摘要

摘要： Graph Neural Networks (GNNs) and Transformers share significant similarities in their encoding strategies for interacting with features from nodes of interest, where Transformers use query-key scores and GNNs use edges. Compared to GNNs, which are unable to encode relative positions, Transformers leverage dynamic attention capabilities to better represent relative relationships, thereby becoming the standard backbones in large-scale sequential pre-training. However, the subtle difference prompts us to consider: if positions are no longer crucial, could we substitute Transformers with Graph Neural Networks in some fields such as Single-Cell Transcriptomics? In this paper, we first explore the similarities and differences between GNNs and Transformers, specifically in terms of relative positions. Additionally, we design a synthetic example to illustrate their equivalence where there are no relative positions between tokens in the sample. Finally, we conduct extensive experiments on a large-scale position-agnostic dataset-single-cell transcriptomics-finding that GNNs achieve competitive performance compared to Transformers while consuming fewer computation resources. These findings provide novel insights for researchers in the field of single-cell transcriptomics, challenging the prevailing notion that the Transformer is always the optimum choice.